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I. 


INTRODUCTION 


A. BACKGROUND 

During a time period when the diseussion of defense budget cuts and 
sequestration dominates the pages of national and defense news, what strategy is the 
military employing to operate more efficiently? The wars in Iraq and Afghanistan are 
coming to a close, and with that, a further reduction in operating budgets is expected 
across the Department of Defense (DoD). The need for a prudent and accountable 
expenditure of taxpayer dollars has reached its pinnacle with the requirement for the DoD 
to be 100% auditable by 2017, per Secretary of Defense Panetta’s direction. The priority 
of fiscal responsibility has gone so far as to be embedded in the National Security 
Strategy, as stated in the following excerpt: 

Cost-effective and efficient processes are particularly important for the 
Department of Defense, which accounts for approximately 70 percent of 
all Federal procurement spending. We will scrutinize our programs and 
terminate or restructure those that are outdated, duplicative, ineffective, or 
wasteful. The result will be more relevant, capable, and effective 
programs and systems that our military wants and needs. (President of the 
United States, 2010, p. 34) 

In addition to spending funds more wisely, further motivation behind this research 
is to improve the overall condition of readiness experienced across the Navy’s fleet. 
Readiness is measured in various ways, ranging from detailed ship-wide assessments to 
the submission of an individual casualty report (CASREP). A CASREP is a report made 
to the ship’s operational chain of command concerning a significant equipment 
malfunction that cannot be corrected within a 48-hour period. Due to the frequency and 
specific nature of CASREPs, these reports played a significant role in the analysis 
discussed in this paper. 

In a 2012 report, the Government Accountability Office (GAO) provided 
evidence, with results displayed as follows in Eigure 1, concerning a noticeable increase 
in the number of CASREPs reported from 2008 to 2012; as the GAO noted, an increase 
in the number of CASREPs would indicate a decline in overall material readiness (GAO, 
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2012). In our research, we used CASREP frequency and severity as a measure of 
effectiveness, and most important, we considered only those CASREPs that require 
additional repair parts due to onboard allowance shortages. 



Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 Q2 Q3 Q4 Q1 
2008 2009 2010 ' 2011 2012 

■ Surface Combatants ■ Amphibious Warfare Ships 


Figure 1. Average Number of Casualty Reports of Surface Combatant and 
Amphibious Warfare Ships by Quarter From January 2008 Through March 2012 

For the U.S. military to remain effective as funding is reduced, efficiencies and 
cost-saving opportunities must be identified and implemented to ensure that the nation 
can sustain a competitive advantage in the 21st century. There are myriad ways that the 
DoD can achieve cost savings from the numerous programs in development, as well as 
from those that have reached maturity and are currently in the sustainment phase. In this 
research, we seek to identify one such avenue for the U.S. Navy that would require a 
minimal investment and result in maximum effectiveness. 

In addition to the fact that the DoD has been mandated to reduce spending and 
find efficiencies wherever possible, other significant reasons justify why such research 
should take place. Following the events of September 11, the entire DoD saw a dramatic 
increase in the amount of funding appropriated for operations and maintenance (O&M; 
0MB, 2003). This O&M funding is used to pay for the day-to-day activities of our 
deployed or deployable forces and typically consists of the costs associated with fuel. 
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repair parts, and maintenance. The increase in O&M funding resulted in a decrease in 
acquisition funding that covers the research, development, and production of the next 
generation of weapons systems. As the campaigns in Afghanistan and Iraq expire, O&M 
funding is expected to decrease and return to more traditional, peacetime levels. 
However, the expected decrease in O&M funding does not guarantee a subsequent 
increase in procurement funding. This distinction is important to note, since the findings 
of the present research may impact both appropriation types and, if successful, would 
result in a reduction of total ownership cost (TOC). 

According to the Defense Acquisition University (DAU, 2012), TOC is defined as 
a concept designed to determine the true cost of the design, development, ownership, and 
support of DoD weapons systems. Within TOC are the acquisition costs and operations 
and sustainment (O&S) costs of the system, with the O&S costs accounting for a 
significantly larger segment. Figure 2 presents the breakdown of TOC of a weapons 
system with greater clarity. 



Figure 2. Total Own 

ership Cost Throughout the System Life Cycle 
(From Defense Systems Management College, 1997, p. 181) 
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Figure 2 makes clear, and the GAO has agreed, that significantly more cost 
savings can be generated throughout the O&S phase as a result of more aggressive 
research during the procurement phase of the system life cycle (GAO, 2010). A 
reduction in O&S costs translates directly to a reduction in the amount of O&M funding 
required for agencies to operate, which further results in an overall savings by the DoD. 
As mentioned previously, a reduction in O&M funding would provide the opportunity to 
invest more in the acquisition of critical programs and potentially improve our national 
defense capabilities. Our research focuses on the O&S phase, since this is the phase that 
requires the largest amount of funding. 

B. SCOPE 

The scope of this project is to conduct a business case analysis (BCA) of the 
Continuous Integrated Logistics Support-Targeted Allowancing Technique (CILS—TAT) 
with the assistance of the Naval Sea Logistics Center (NSLC). According to the NSLC 
command description, “NSLC is a field activity of the Naval Undersea Warfare Center, 
and is tasked with providing integrated logistics, engineering, and information technology 
expertise to all facets of the Navy's worldwide logistics support structure” (NSLC, 2012). 

Embedded in NSLC’s integrated logistics support responsibilities are the 
requirements to manage the configuration of naval weapons systems within the different 
classes of ships and submarines as well as to provide the appropriate allowances for spare 
parts to be stocked onboard. Configuration management, while closely related to the 
allowancing process, is beyond the scope of our research and therefore is not fully 
introduced here. Our research instead focuses on a subset of the allowancing process 
known as CILS-TAT. We discuss this technique, as well as the traditional allowancing 
process, in much greater detail in subsequent chapters and provide a better understanding 
of the variables considered when NSLC is determining allowances. 

Our current motivation behind evaluating the effectiveness of the CILS-TAT is 
the suspension of the program since 2009, when the U.S. Navy began its transition to an 
enterprise resource planning (ERP) framework. We expect that by the time this research 
is published, the U.S. Navy’s allowancing process systems will have fully transitioned 
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into the ERP framework; therefore, the time for a possible re-implementation of the 
CILS-TAT is quiekly approaehing. Based on our findings, we hope to demonstrate the 
level of effeetiveness aehieved as a result of using the CILS-TAT and provide 
reeommendations eoneerning the use of the program in the future. 

The CILS-TAT ean be used onboard any U.S. Navy platform. To further narrow 
the seope of our projeet, we foeused on 18 ships from the U.S. Navy’s Ballistie Missile 
Defense (BMD) Lleet. Live Tieonderoga-elass guided missile eruisers (CGs) and 
13 Arleigh Burke-elass guided missile destroyers (DDGs) make up our sample. Critieal 
to the U.S. National Seeurity Strategy, the mission of the BMD Lleet is to deteet, traek, 
and intereept ballistie missiles of all ranges and types (Missile Defense Ageney, n.d.). 
The prineipal reason for our seleetion of these ships is not only the importanee of the 
BMD mission but also the eommonality of systems eontained within the Tieonderoga and 
Arleigh Burke elasses. With the Arleigh Burke elass still in produetion, the results of this 
researeh will provide the longest period of applieability and support the largest elass of 
ships in the naval inventory. 

C. PROBLEM DESCRIPTION 

The task of minimizing eosts while maximizing effeetiveness is not one that ean 
easily be aehieved and, in some eases, may not be realistie. The goal of this projeet, 
however, is to do just that through an aggressive spare-part allowaneing model that will 
improve the effeetiveness of the U.S. Navy surfaee fleet through a reduetion in O&S 
eosts. The projeet analysis eovers two distinet time periods to eapture the true value of 
the CILS-TAT and then uses foreeasting models to determine the future viability of the 
program. The first period eovers the years 2003-2009. During this time, 18 CILS-TAT 
reviews were eondueted and 48 months of operational data was available per ship. 
During the seeond period, from 2009-2011, no CILS-TATs were eondueted and no other 
allowaneing maintenanee was done. Based on the data available for analysis during these 
periods, we attempted to foreeast the long-term effeetiveness of the CILS-TAT. 
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Unlike land-based forees, where the addition of another warehouse to store repair 
parts is not a major eoneem, on a U.S. Navy ship, there is a finite amount of spaee to 
store repair parts. In a perfect world, a ship could carry a replacement part for every 
installed component in the event that a repair is required. Since that is not feasible, a 
great deal of consideration must be given to determine the right mix of parts that make up 
the ship’s onboard allowance list. A shipboard allowance list for maintenance parts is the 
larger list from which spare items are selected, or not selected, for onboard allowances. 

In the past, one problem with such lists has been assuring maximum value 
received for dollars spent on spare items. Such value may only be received by making 
sure that spare items are ordered through some type of a combinatorial optimization 
process (Harrahy, Powell, & Lutz, 1968). According to the GAO (2003), the U.S. 
Navy’s spare-parts supply problems can delay the completion of needed maintenance and 
repair jobs on deployed ships and can affect their operations and mission readiness. In 
the same 2003 GAO report, covering two carrier strike groups over six deployments, 
58% of the 50,000 maintenance work requests were delayed because the appropriate parts 
were not located on the ships. It can be deduced from these studies that getting the right 
mix of parts onboard a ship is a costly, complex, and critical task with far-reaching 
implications. The processes involved in allowancing have evolved over time. We cover 
two such processes in subsequent chapters. 

Over the course of several decades, initiatives have been implemented to either 
improve the business practices associated with allowancing or reduce slow-moving and 
unnecessary inventory (K. R. Bitner, personal communication, November 5, 2012). 
Allowances can be reduced for a number of reasons ranging from obsolescence to a lack 
of demand from the system. When an allowance is reduced due to a lack of demand, 
greater risk is incurred in the event of an equipment casualty. The severity of a casualty 
can range from an insignificant impact to the loss of a critical system and failure to 
complete a primary mission. These circumstances obviously cover a wide spectrum but 
hopefully provide some insight into the factors that must be considered at the component 
level when determining allowances. 
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One proposal to help the U.S. Navy more efficiently use its appropriated funds 
has been to spend less in the procurement of spare-parts allowances onboard ships. The 
problem with reducing the number of parts onboard is that when a system fails and the 
parts are not onboard, logistics support is required—whether the part is coming from the 
other side of the globe or from a warehouse a few miles away. In addition to the required 
logistics support, the system requiring the part is either non-operational or degraded 
while awaiting repairs. The costs of the part not being onboard can be measured 
quantitatively in monetary terms and more abstractly by a decrease in the ship’s 
capabilities. 

The monetary costs of providing material support to an operational vessel are the 
most simple to compute and typically involve determining the cost of the part required, 
the location of the vessel requiring it, and the urgency of the need for the item. 
Transportation costs are reasonably easy to obtain and can be forecasted for a variety of 
scenarios using simulations. The more difficult cost to capture is the significance of 
losing a mission-critical system for an extended period as a result of not having the parts 
in the right place at the right time. 

D. RESEARCH OBJECTIVES 

The objectives for our research include the following: 

1. Conduct a BCA of the CILS-TAT process using historical data. 

2. Measure the costs associated with performing a CILS-TAT. 

3. Provide recommendations to NSLC concerning possible improvements to the 
existing CILS-TAT model. 
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E. RESEARCH QUESTIONS 


We have addressed the following questions in our researeh: 

1. How did CILS—TAT irnpaet allowanee effectiveness for our sample, and 
at what cost? 

2. Are there correlations between allowance effectiveness rates, percentage 
of time free (POTF) from CASREPs, and CILS—TAT cost? 

3. What was the effect of not having CILS—TAT during 2009-2011, when 
traditional allowancing procedures were not available? 
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II. BACKGROUND 


A. ALLOWANCING FUNDAMENTALS 

When considering the austere environment within which U.S. Navy ships operate 
and the absolute necessity to function above readiness metrics, it is important that the 
U.S. Navy develop and implement a robust set of allowancing procedures to ensure that 
the right parts are on the shelf when equipment fails. As related to inventory 
management, allowancing can be defined as determining the correct blend of items 
carried onboard to ensure that the part is available to restore the equipment when failures 
occur. The correct quantity of items is influenced by constraints such as funding and the 
cost of procurement, the availability of the item, space availability onboard the ship, 
mean time between failures (MTBFs), and estimated lead-times for replenishment. 

As mentioned previously, the entire purpose of the allowancing process is to 
generate the right set of parts to stock onboard the ship, thereby ensuring that the overall 
mission readiness levels of the vessel do not drop below prescribed metrics set forth by 
the U.S. Navy. To measure the effectiveness of the allowancing process, the U.S. Navy 
has formulated several performance metrics, such as operational availability (Ao), supply 
gross issue effectiveness, supply net issue effectiveness, CASREP frequency, and POTF 
from CASREPs. 

1. Operational Availability 

Operational Availability (Ao) provides a method of predicting and assessing 
system performance and readiness during the acquisition process and then becomes the 
performance benchmark during initial operational capability (IOC), deployment, and 
operations/maintenance cycles (Chief of Naval Operations [CNO], 2003). The U.S. 
Navy’s ability to meet the highest readiness levels is principally derived by the Aq of the 
warfighting systems and equipment installed onboard U.S. Navy vessels. 

According to Chief of Naval Operations Instruction (OPNAVINST) 3000.12A 
(CNO, 2003), the calculation for determining the Ao is a probability function of 
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reliability, maintainability, and supportability components. The equation for determining 
Ao is written as the following; system up time divided by total time (up time + down 
time). 

Operational Availability is fundamentally considered as a supportability 
calculation of the equipment/system in terms of predicted reliability (R), called mean 
time between failure (MTBF); predicted maintainability (M), in terms of mean time to 
repair (MTTR); and designed supportability, called mean logistics delay time (MLDT; 
CNO, 2003). Inherent availability (Ai), an alternative metric, does not include a 
consideration of support functions related to re-supply transportation and repair. Once Ai 
is determined, you are ready to add in the supportability portion, or MLDT, to calculate 
Ao (CNO, 2003). See Figure 3 for a graphical representation of Ai and Aq. 


A> = 

MTBF 


MTBF + MTTR 

MTBF 

Ao- 

MTBF + MTTR 

+ MLDT 


Hardware/Software 
Design Considerations 

Logistics System 
Design Considerations 


Figure 3. Logistics Impact on Operational Availability 

(From CNO, 2003) 


2. Measuring Allowancing Effectiveness 

In order to effectively measure the allowancing packages that the U.S. Navy is 
developing and sending to the fleet, they have constructed several calculations to 
interpret the demand signal against the assets onboard. The metrics are assembled by 
segregating the demand into different pools or supply source codes based on the stock 
posture of the requirement. Figure 4 represents the various codes. 
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CODE DEFINITION 

A Allowance List Material issued from storeroom stock 
C Non-Allowance List Material (SIM) issued from storeroom stock 
D Allowance List Material Not In Stock (NIS) when requested 
p Non-Allowanee List Material (SIM) Not In Stock (NIS) when 
requested 

Not Carried (NC) repair parts which are not listed on an APL in the 
ship's COS file 

j Not Carried (NC) repair parts which are listed on an APL, but 
does not compute for an allowance 

Figure 4. U.S. Navy Allowance Supply Source Codes 

(From Naval Supply Systems Command [NAVSUP], 1997) 

The calculation used in evaluating allowance performance is the allowance 
effectiveness metric, which is represented by the following equation; (^A + C + D + 
F)/(^A + C + D+ F + G+J). The allowance effectiveness calculation allows 
stakeholders to determine the strength of the allowanced items as well as the non¬ 
allowance material selective item maintenance (SIM) against demand (NAVSUP, 1997). 

3. Supply Issue Effectiveness Rates 

In an effort to measure the overall issue effectiveness of the allowancing packages 
throughout the training cycle and into deployment, the U.S. Navy has developed 
a set of inventory issue effectiveness rates to judge the ability of the inventory to 
meet the demands placed against that inventory. There are two metrics that are used to 
determine the success of the allowancing package, the first being the supply gross issue 
effectiveness rate. Supply gross issue effectiveness measures the number of requirements, 
commonly called requisitions, issued against the total number of requirements. Supply 
gross issue effectiveness is computed by dividing the number of requisitions issued by 
the total number of requisitions; (i4-l-C)/(i4-l-C-l-D-l-F-l-(7-l-/). 

The aforementioned performance measurement allows the managers of the 
allowancing package to quantitatively judge whether or not they have the correct variety 
and quantity of parts on the ship. The calculation provides a percentage value for every 
demand that was issued from the storeroom. In other words, it reveals how many parts 
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were not onboard that should have been onboard given the current demand. The output 
from this calculation would provide an allowancing package stakeholder with a sense of 
how many parts or line items must be added to the allowancing package to meet issue 
effectiveness goals. 

The second metric that the U.S. Navy uses is the supply net issue effectiveness 
rate. This measurement tool is calculated by dividing the total number of issued 
requisitions by the total number of requisitions minus the number of requisitions not 
carried (NC) onboard; (^A + C)/(^A + C + D + F). This more focused rate allows the 
stakeholders to obtain a sense of how the variety and quantity of the parts included in the 
allowancing package are reacting to the current demand signal. For the purpose of 
allowancing package development, the use of the supply gross issue effectiveness rate 
would prove to be more useful since stakeholders desire to know on average how well the 
allowancing package supported the demand as a whole. 

4. Casualty Reporting 

A measure of effectiveness central to our project involves tracking the number 
and severity of CASREPs for each of the 18 ships in the sample. A CASREP is a report 
made to the ship’s operational chain of command concerning a significant equipment 
malfunction that cannot be corrected within a 48-hour period. The release of a CASREP 
will also alert supporting activities of the urgency to provide aid in the form of spare parts 
or technical assistance. The significance of each CASREP is determined by the impact of 
the equipment failure on the ship’s mission. There are different categories of CASREPs, 
ranging from Category Two (C2) through Category Pour (C4), with C4 being the most 
severe. A C4 CASREP denotes that a deficiency exists in mission-essential equipment 
that causes the loss of at least one primary mission. A C3 CASREP denotes that a 
deficiency exists in mission-essential equipment that causes a major degradation but not 
the loss of a primary mission. Pinally, a C2 CASREP indicates that the ship has lost 
redundancy in one of its primary or secondary mission areas. CASREPs are required to 
be updated periodically by the ship until the damaged equipment is restored to a fully 
mission-capable state. The frequency of updates is correlated to the severity of the 
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CASREP. The CASREP will remain an open report until all repairs are finalized 
(NAVSUP, 1997). 

5. Percentage of Time Free From CASREPs 

POTE is a readiness metric tracked by NAVSUP Corporate Information Systems 
(CIS) that records the number of days in a reporting period that a ship does not have an 
open C3 or C4 CASREP. This metric is a central measure of a ship’s material readiness 
and one that we use in the analysis section of this report. A calculation of time free of 
casualties is 


2ji=i 


100 X 


|^(^days out of shipyard and free 


CASREPS 


during period)\ 


(Calendar days during period) 


1 . 


( 1 ) 


where 


i = a running index for individual ships in the grouping that have been 
active for more than 1/5 of the calendar period and 

K = total ships in the grouping meeting the active time criteria. 

B. ALLOWANCING STAKEHOLDERS 

The entire allowancing process is completed by a vast array of logistics and 
maintenance professionals employed by the U.S. Navy and civilian partners. The 
requirements necessary to develop a comprehensive allowancing package deem that all 
influencers collaborate effectively as one cohesive team. The requirements for the 
allowancing package range from ship configuration data, maintenance and logistics data, 
and if available, logistics support data specifically related to reliability concerns. 

On the logistics side of the house, there are a couple of key players across the 
U.S. Navy. The first of these players—and probably the most dominant in calculating the 
allowances—is the NAVSUP Weapons System Support, Mechanicsburg (WSS Mech). 
They are the U.S. Navy’s representative to manage the allocation of resources relating to 
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the allowancing process. NAVSUP WSS Mech possesses the models necessary to 
develop allowancing packages and work with all stakeholders to ensure that the end 
product fits the needs of the warfighter. In addition, they are tasked with controlling the 
funding for initial outfitting onboard the vessels. 

On the maintenance side of the house, the stakeholder most concerned with the 
development of the allowancing process is the Naval Sea Systems Command 
(NAVSEA). They are tasked with ensuring that the ship is configured properly with the 
most up-to-date systems and equipment. To ensure that the configuration of all systems 
is correct, NAVSEA works closely with NSEC, who provides integrated logistics 
support. The configuration of the ship must be accurate when developing the 
allowancing package to facilitate superior sustainability during the entire life cycle of the 
ship. 

When developing a product, you must have a customer. In this case, the customer 
is the warfighter on the waterfront: Commander, Elect Eorces Command (CEEC). CEEC 
is charged by the CNO to ensure that all fleet forces maintain the highest levels of 
operational readiness to support the nation’s maritime strategy. With regard to surface 
ships. Commander, Naval Surface Eorces (SUREOR) is the stakeholder involved in the 
allowancing process and subsequent reviews of that process as the ship progresses 
throughout its life cycle. Within SUREOR, there are maintenance and logistics 
components that work in concert with their counterparts involved in the process. 
Eundamentally, once SUREOR takes ownership of the vessel, it is ultimately responsible 
if the ship does not meet the mission. 

The maintenance personnel at SUREOR manage the maintenance practices and 
configuration changes related to keeping the ship up to date. The logisticians are 
constantly monitoring the issue effectiveness rates to determine if an allowancing 
package should be applied to increase the mission readiness. 

C. FUNDING 

The allowancing process requires two different classifications of funding to pay 
for all inventory required: initial issue provisioning and follow-on replenishment over the 
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life cycle of the allowance being tied to that vessel. The reasoning behind using two 
different classifications of money is the delineation between the Budget Activities (BA) 
of Operations and Maintenance, Navy (0&M,N) and Other Procurement, Navy (OPN) 
within the DoD budget. The two classifications of money are controlled by separate 
organizations within the U.S. Navy. 

The funding stream more closely tied to the allowancing process and initial 
provisioning is the OPN appropriation. This appropriation is designated for the 
procurement, production, and modernization of support equipment and materials not 
otherwise provided for as well as the procurement and installation of equipment 
(Department of the Navy [DoN], 2012). Because of the designation of these funds to 
support the initial outfitting of naval forces, all initial provisioning within an allowancing 
package are paid for with OPN funds. 

The allocation of these funds from Congress is given to NAVSUP WSS Mech to 
determine the best allocation across the competing priorities within the enterprise. The 
funds are then designated as OPN-8 funds. The Navy Working Capital Fund (NWCF) 
then buys spare parts listed in the new allowancing product with OPN-8 funds following 
the post-material support date (MSD; CNO, 2012). Once the allowancing products are 
developed with the inputs from stakeholders, NAVSUP WSS Mech makes the 
determination to purchase the requisite spare parts to fill the allowances. 

The appropriations assigned to the BA of 0&M,N are designated to finance the 
day-to-day costs of operating naval forces, including fuel, supplies, and maintenance of 
ships, U.S. Navy and Marine Corps aircraft, related weapons systems, and the support 
establishment ashore (DoN, 2012). Consequently, any replenishment for stock that the 
ship might require to maintain operational readiness must be paid for with 0&M,N 
dollars. At the force level, these funds are controlled by the type commander (TYCOM) 
and allocated down to the unit supply officer on a quarterly basis in the form of an 
operating target (OPTAR). The OPTAR will pay for all spare-part requirements and any 
related costs, to include transportation costs, ordering costs, and any holding costs 
associated with keeping the part in inventory. 
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D. ALLOWANCING PRODUCT MAKEUP 


1. Depot-Level Repairable Versus Non-Depot-Level Repairable 

The typical allowancing package is composed of a variety of different line items 
and quantities dependent on the type of equipment onboard the ship, the dollar value, the 
required endurance level, and the average MTBF. The most basic classification among 
the varying spare parts in the allowancing package is the designation as a depot-level 
repairable (DLR) item versus a non-DLR item, otherwise known as a consumable item. 

DLR items are usually the high-dollar-value components and are deemed too 
costly to dispose of. This classification requires that when a DLR is unserviceable, it 
must be sent back to the pre-determined repair facility for overhaul and subsequent 
clearance as ready for issue (RFI). When considering the dollar value of these items, the 
U.S. Navy aggressively tracks the movement and repair status of all items both on the 
vessels and ashore to eliminate unnecessary waste. As a rule of thumb, the allowance for 
a DLR in an allowancing package is incapable of being altered by the personnel onboard 
the ship. These allowances are to remain as they were when they were implemented. 
The designation of DLRs is determined by NAVSUP guidance (NAVSUP, 1997). 
Specific codes are assigned to segregate the DLRs by their application of use in a 
particular system and into manageable groupings based on their level of usage. In 
addition, these codes are used to designate which inventory manager will handle the day- 
to-day administration of that particular item. 

The non-DLR items are not tracked as closely but still require a compulsory level 
of oversight. Consumable items are kept onboard the ship to be used as necessary; once 
the life of that asset has been exhausted, the item is simply discarded in accordance with 
published guidelines. The logistics personnel onboard the ships are given authorization 
to manipulate the allowance of these items as they see fit to meet future demand signals. 
The following is a list of the most commonly found allowance classifications for non- 
DLR items: 9C, 9B, 9G, 9N, and 9E (NAVSUP, 1997). 
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2 . 


Allowance Type Codes 


When apportioning allowances, the allowancing package will assign a 
classification code to the assets within the package. This will enable the logistics 
personnel to know by which process the asset was placed on the ship. The classification 
codes are broken down into nine different categories, called allowance type (AT) codes. 
The most common classification with regard to allowancing products is an ATI. This 
classification explains that the asset is mandated to be carried because of the 
implementation of an allowancing product. A breakdown of the other applicable AT 
codes and descriptions are captured in 


Table 1. Allowance Type Codes (FromNAVSUP, 1997) 


AT Code 

Name 

Description 

1 

Cosal Item 

COSAL item. 

2 

Aviation Support Item 

Load List item, applicable to load carrying ships only. 

3 

COSAL/AV Item 

Load and Allowance List item, applicable to load carrying ships only. 

4 

Demand Based Item (DBI) 

Non-COSAL item that is stocked based solely on demand. 

5 

TYCOM Directed Item 

Non-COSAL authorized add item based on specific TYCOM authority. 

6 

Excess Item 

Non-COSAL excess item that does not have sufficient demand to maintain. The 

item is to he offloaded and deleted. 

7 

Economic Retention Item 

Economic retention. Excess item that, because of low unit cost (normally under 
$100.00), is authorized for retention until the next ILO/ReAVCAL. The exact 
amount may be specified by each TYCOM. 

8 

Demand Recording 

Non-COSAL item that is established for demand recording only. It wfll he changed 
to AT code 4 if demand reaches established criteria, otherwise it wfll he deleted 
after 24 months with no demand. 

9 

Substitute/Altemate Item 

Assigned to a substitute item which is not stocked as a primary number. 


3. Mission Criticality Codes 

The U.S. Navy’s Item Mission Essentiality Coding (IMEC) system consists of a 
combination of military essentiality code (MEC) and mission criticality code (MCC). 
These codes help the allowancing product developers to apply weights to those assets that 
are most important to the sustainability and readiness of the ship within the allowancing 
model. The allowancing package will contain either the MEC or MCC in Coordinated 
Shipboard Allowance Eist (COSAL) Part I, Sections A and B, depending on the ship’s 
computation method (DoN, 2009). The possible MCCs that can be assigned to 
components are listed in Table 2. 
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Table 2. Mission Criticality Codes (From DoN, 2009) 

CODE DEFINITION 

1 Failure of component/equipment causes minor mission impact. 

2 Failure of component/equipment causes total loss or severe 
degradation of a secondary mission. 

^ Failure of componenCequipment causes severe degradation of a 

primary mission capability. 

Failure of component/equipment causes total loss or severe 
4 degradation of mobility or primary mission (propulsion or life 
support). 

^ The loss of this equipment results in a safety hazard to the ship or its 
crew. 

X Assigned to all Allowance Equipage Lists (AEL). 


E. ALLOWANCING INPUT AND OUTPUT 

I. Inputs 

The process for developing an allowance package requires several pieces of data 
from varying stakeholders within the process. The information necessary to effectively 
determine the allowancing package ranges from configuration management data to 
maintenance history data located within the ship’s Maintenance, Material and 
Management (3M) database. 

The configuration management data is taken from the Configuration Data 
Managers Database-Open Architecture (CDMD-OA), which is managed by NSEC. 
CDMD-OA tracks the status and maintenance of naval equipment and their related 
logistics items (e.g., drawings, manuals) on ships and naval activities around the world. 
The status of a given piece of equipment on a ship determines what and how many spare 
parts will be stored on that ship for that equipment, making this tracking extremely 
important in terms of cost, shipboard space and weight, and the operational availability of 
the ship (CDMD-OA, 2012). The U.S. Navy identifies the required spare parts onboard 
U.S. Navy vessels through the use of allowance parts lists (APLs). APLs provide support 
and outfitting for parts that are required for the particular maintenance action performed 
onboard the ship (Alvarez, 2010). 
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The 3M database is a eentral point on the ship where all data is kept related to 
maintenance performed. In addition, the database contains a comprehensive list of all 
assets that are and should be kept on the ship to perform maintenance. Along with the 
comprehensive list is the associated usage data for the parts. If the ship is a new 
construction, the usage data from similar vessels across the fleet will be used in the 
development of the allowancing package for the new vessel. 

In addition to gathering the maintenance, demand, and configuration data from 
the 3M system, data must also be extracted from the ship’s current stock record file 
(SRF). The SRF contains the financial records of the ship to gauge the impacts of the 
previous allowancing product against the current funding levels. This information can be 
very telling as to whether or not the previous product was effective at saving costly 
procurements and the inclusion of unnecessary carrying costs. 

The final portion of information that must be incorporated into the allowancing 
package is the Weapons System File (WSF) Level C, which contains all provisioning and 
technical decisions. 

2. Outputs 

With the consideration of the aforementioned input of configuration, financial, 
maintenance, and demand data, the stakeholders are capable of applying this information 
into a sophisticated mathematical allowance model that provides the optimized 
allowancing package. The model can be manipulated to varying degrees, permitting 
different types of allowancing packages dependent on the desires of the stakeholders. In 
the next section, we explore several different types of allowancing packages. 

Once the model produces the output, the package is socialized among the 
stakeholders. Once it is finalized, the allowances are generated by NAVSUP WSS Mech 
and funded and released to SURFOR for issuance to the ship. The final piece of the 
puzzle is that the ship will integrate the new product into the ship’s database and drop the 
order requisitions to stock the shelves accordingly. 
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F. TYPES OF ALLOWANCING PACKAGES 

1. Comprehensive Allowancing 

As previously discussed, the output product from the allowancing process can be 
tailored to focus on a particular weapons system or set to review the entire ship’s spare 
parts support package. The most common comprehensive allowancing package is the 
COSAL. The COSAL provides both technical and supply information, which makes it an 
integrated logistics support (ILS) document. It is a technical document to the extent that 
equipment/component/part nomenclatures, operating characteristics, technical manuals, 
etc. are described in APLs or allowance equipage lists (AELs; NAVSUP, 1997). 

Implementation of the COSAL is typically conducted during the initial outfitting 
of the platform at the beginning of its life cycle. Factors determining the composition of 
the COSAL are the maintenance philosophy/capability, support concept, logistics 
response time, historical demand, and item/system population and redundancy. Since the 
COSAL is the primary listing for all spare parts allocated for the ship, logisticians 
routinely reference the COSAL in their day-to-day operations. Purportedly, the COSAL 
is designed to provide the warship with a sustained level of material support for 90 
combat days without replenishment and is tailored to a particular ship class, because each 
ship class has a different weapons system configuration (Axinto & Giles, 2005). 

Once implemented onboard the ship, the COSAL begins to receive periodic 
updates to support any configuration or allowancing modifications. Because COSALs 
are costly to implement and very time intensive, the U.S. Navy has developed a system of 
implementing the incremental changes to COSALs by way of an automated shore 
interface (ASI). These updates are specifically tailored to a particular ship or class to 
ensure that the COSAL remains up to date with configuration changes or additions and 
deletions to the allowancing package. The periodicity of ASI updates varies depending 
on the timing of the changes included in the updates and the funding levels available to 
support those changes. The process of using ASIs is far less costly and less labor 
intensive on the ship and the shore facilities. 
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2. Targeted Allowancing 

Considering the signifieant eosts involved with developing and implementing a 
COSAL, the U.S. Navy has worked towards minimizing those eosts by employing a 
targeted allowaneing proeess. Stakeholders use this foeused approaeh to zero in on the 
spare-parts requirements that would provide the greatest benefit towards inereased 
readiness of the shipboard allowancing package. The process of targeted allowancing 
follows a similar path to the comprehensive review, with one exception. A filter is 
applied to the mathematical model that only lets certain aspects of the COSAL be 
reviewed based on criteria set forth by the stakeholders. Once the targeted allowancing 
package is produced, the implementation process is identical and the funding follows a 
similar stream for procurement of the assets. 

G. CONTINUOUS INTEGRATED LOGISTICS SUPPORT-TARGETED 

ALLOWANCE TECHNIQUE (CILS-TAT) 

I. Overview 

One of the common targeted allowancing techniques that the U.S. Navy has 
adopted is the process termed the CILS-TAT. A CILS-TAT is used to focus the 
allowancing process to minimize the overall funding requirement, increase support for 
poor-performing equipment, and reduce allowance churn while providing the greatest 
increase to readiness for the ship. In the current fiscally constrained environment, this 
approach is absolutely vital to ensuring that the dollars are spent optimally. 

The U.S. Navy’s Maritime Allowance Working Group (MAWG) began 
discussing the use of the CILS-TAT in 1999 with 15 prototype ships. The concept was 
to develop a technique by which ship-optimized allowances were discreetly applied to 
update a ship’s SRF, eliminating allowances with no usage and targeting systems with 
usage for allowance updates of either range additions or depth increases. During the 
prototype phase, the MAWG measured success by looking at the increase in supply 
effectiveness, reduction in cost, and churn of the allowances. Through evaluation of 
these metrics specifically, they recognized a 47% and 50% reduction in the line-item 
chum and cost of new allowances, respectively, during the prototype phase (Bmno, 
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1999). The results of the prototypes also revealed a 0.9% inerease in the supply gross 
issue effeetiveness, from 51.8% to 52.7% (Bruno, 1999). 

From 1999 to 2009, the U.S. Navy adopted CILS-TAT as one of the primary 
methods for eondueting targeted allowaneing and has implemented the proeess on many 
different platforms, to inelude Los Angeles-elass fast-attaek submarines (SSNs), Arleigh 
Burke-elass DDGs, Avenger-elass mine eountermeasures ships (MCMs), Oliver Hazard 
Perry-elass frigates (FFGs), and Tieonderoga-elass eruisers (CGs). The seleetion proeess 
for implementation of a CILS-TAT on a ship is eondueted by various stakeholders 
ineluded in the MAWG, to inelude representatives from NSLC, NAVSUP WSS Meeh, 
NAVSEA, and the TYCOM. 

2. Time Line 

The time line for implementation typieally follows the deployment eyeles and is 
displayed in Figure 5. After a ship has eompleted two years of normal underway 
operations, data is eolleeted from that period and evaluated by the MAWG to determine 
if the ship is a eandidate for a CILS-TAT. As they assess the data, several key faetors 
influenee the deeision made by the MAWG, ranging from the availability of time for 
implementation (i.e., maintenanee availability opportunities), a reeognized reduetion of 
the supply gross issue effeetiveness, a deerease in readiness levels below the fleet 
average, and funding availability. 



Figure 5. CILS-TAT Implementation Time Line 
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Once the CILS-TAT is eompleted and ready for implementation, the produet is 
given to the TYCOM, who is then assigned the responsibility of working with the 
operational unit to implement the produet into the ship’s onboard databases. In an effort 
of eontinuous evaluation, at the eonelusion of another two-year period, the MAWG will 
onee again review the health of the ship and determine if the proeess was profitable and if 
another CILS-TAT produet should be introdueed. 

3. Input 

In order to develop the CILS-TAT, produet stakeholders draw from several 
different systems and add a filter into the proeess to foeus on a narrowly defined set of 
data. The systems used in the CILS-TAT are the same as those used to formulate a 
eomprehensive allowaneing produet and inelude the CDMD-OA, 3M data from the ship, 
CASREP information, and the WSF. Onee this information is eompiled, it is run through 
an elaborate mathematieal allowaneing model. The final pieee in the applieation is a 
filter that foeuses the allowaneing paekage to only look at a partieular type of 3M data. 
Onee the filter is applied, the output fide is generated and reviewed. The CILS-TAT’s 
filter only (a) allows new allowanees to APLs that have had 3M usage, (b) provides 
allowanee deletions or deereases for those APLs with no reported usage, or (e) provides 
deletions for parts that no longer support installed eonfigurations. This filter is what 
makes this allowaneing produet a more targeted approaeh. 

4. Output 

The output provided by the model is an allowaneing paekage that truly foeuses on 
improving the eolleetion of parts neeessary to support the ship based on the demand that 
has been realized over the previous two years. Those systems that have needed a 
partieular asset onboard but the part was not available would now be added to the ship’s 
inventory, thereby improving the supply gross issue effeetiveness and operational 
readiness. Speeifieally, the output file would inelude (a) range additions to prevent 
missing a future demand and (b) range and depth deereases. Those parts that historieally 
have not been demanded or do not eurrently eonform to the eurrent eonfiguration 
onboard would be offloaded through attrition. This more foeused approaeh truly reduees 
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the cost of allowancing, especially when you calculate the cost of allowancing over the 
entire U.S. Navy fleet while providing the most efficient use of dollars to improve overall 
operational readiness. 

5. CILS-TAT Process Flow 

The actual process of a CILS-TAT begins with the collection of 24 months of 
operational data (see Figure 5). The historical demand data is then added to the ship’s 
current COSAL and analyzed for allowance additions and decreases. CILS-TATs are 
ship specific, meaning that no two ships will experience exactly the same types or 
frequency of material demands. There is a degree of commonality among demands, but 
due to different missions performed, environmental factors experienced in the area of 
operation, and the overall material condition of the ship, these CILS-TATs have to be 
performed on an individual basis. 

Once the 24 months of data is drawn down from the ship, the demands registered 
for scheduled and unscheduled maintenance are analyzed. A CILS-TAT will also add 
new allowances when a system configuration change is in process or expected to happen 
in the near future. Allowances will also be added to the SRF in anticipation of required 
maintenance for each of those new systems. 

Once analysis of both the historical demand data and forecasted equipment 
upgrades are complete, the new allowances and the remainder of the COSAL product is 
sent to the ship for loading into the central database. The CILS-TAT output is then 
validated by comparing the total list of national item identification numbers (NIINs) 
recommended by the CILS-TAT against the allowances that the ship already has 
onboard. Any shortfalls identified during this validation become candidates for 
procurement. To simplify this with an example, a CILS-TAT may recommend that the 
ship carry 5,000 different repair parts at a cost of $2.5 million but, after validating the 
CILS-TAT against the ship’s current onboard allowances, determines that only 300 new 
parts at a cost of $250,000 may be additionally required. A CILS-TAT not only 
recommends new allowances but also allows the ship to validate its current configuration. 
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Once all shortfalls are identified, requisitions are prepared for eaeh new NIIN to 
be proeured. The ship is not required to prioritize the new allowanee requisitions sinee 
all additional allowanees are eonsidered valid by NSLC and will be further evaluated by 
the supply system during the requisition filling proeess. There are three distinet phases of 
further evaluation that eaeh requisition goes through before funding is obligated. We 
eover these phases in the next seetion. 

6. Requisition Screening 

Onee requisitions are released into the supply system as a result of a CILS-TAT, 
they are reviewed by the NAVSUP Outfitting Support Aetivity (OSA) and Outfitting 
Requisition Control Aeeounting System (ORCAS) managers. The purpose of this 
sereening is to eonduet quality eheeks and ensure that the eorreet fund eodes and adviee 
eodes were applied to eaeh requisition by the ship. These eodes play an important role in 
determining the right type of funding applied to eaeh of the requisitions and eonfirming 
that the obligations and expenditures eventually mateh for eaeh. Requisitions flow freely 
through this proeess and are not held up for funding eonstraints. The main eoneern here 
is to ensure that the requisition was submitted properly by the ship. 

Onee the requisitions eomplete the quality assuranee proeess, they then move to 
the Pushed Material Inventory Control System (PMICS) offiees, whieh are a braneh 
within NSLC. The new allowanees reeommended by the CILS-TAT are eompared 
against allowanees that are already being proeured via other means—for example, the In- 
Serviee Engineering Aetivity (ISEA). A eontraetor may already have the responsibility 
to provide initial outfitting support for a partieular system or group of systems onboard 
the ship. The sereen through PMICS is done to look at all possible avenues from whieh 
this support eould eome and ensure that the government is not paying for the same initial 
allowanee for the same ship twiee. Requisitions ean sometimes be held up in this proeess 
if the ship is entering a maintenanee availability or shipyard period and material 
deliveries to the ship are being routed to another loeation. 

Onee a requisition elears the PMICS and NAVSUP OS A sereens, requisitions are 
split between high and low value. (High value is any requisition with an extended value 
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of more than $300.) During periods when defense budgets were not so tight, low-value 
requisitions would then immediately be funded and, if material were available, they 
would be sent to the ships. High-value requisitions are caught, and a re-evaluation of 
each allowance is conducted to ensure that the allowance is still valid before funding is 
applied. Since the process up to this point can take up to 45 days, it is possible that a new 
allowance identified by the CILS-TAT may have become obsolete due to an onboard 
system upgrade or a NlfN supersession. The NSLC may also cancel a requisition during 
this phase if the re-evaluation determines that the part in question no longer computes for 
an allowance or that the part has already been provided by the NSLC or the ISEA. 

If a requisition is cancelled in error, the ship is required to contact the NSLC or 
the item manager directly to have the requisition reinstated. At the time of this report, all 
requisitions, both high and low value, generated as a result of a CILS-TAT are initially 
given a status code of “NM” by NSLC, although only high-value requisitions are re¬ 
evaluated. The NM status code signifies that the requisitions are being delayed by NSLC 
until funding becomes available. 

7. Prioritization of Requisitions 

Once funding becomes available for these new allowances, it is applied and 
material can start moving to the ship. NSLC does not determine which requisitions are 
funded or which CILS-TATs are funded first. There may be CILS-TAT requisitions for 
more than one ship that are awaiting funding, and therefore, a determination has to be 
made concerning which CILS-TATs are funded first. As previously stated, the 
determination of priorities at this level is done by recommendations from the TYCOM. 
The TYCOM typically sets its priorities based on which ship is set to deploy next and the 
relative importance of the mission that it is set to carry out. CILS-TAT requisitions are 
funded at this point on an all-or-nothing basis, meaning that if funds are available for an 
entire CILS-TAT output for a particular ship, they are released. Otherwise, requisitions 
continue to hold with an NM status until additional funds are available. No prioritization 
exists for the types (or criticality) of a particular part over another, and as a consequence. 


26 



allowances that have little impaet on overall system readiness (as determined by the 
MCC) may be funded ahead of those that eould eripple a primary mission area. 

H. NAVY ENTERPRISE RESOURCE PLANNING 

1. Overview 

In an effort to improve the funetionally and eompatibility of sueh a wide 
eonglomerate of information systems aeross the U.S. Navy enterprise, senior leaders 
endeavored to find an information software package that eould help the U.S Navy 
streamline the proeess of logisties information sharing. They were able to work with 
eommereial vendors to develop a system that mirrors the civilian equivalent of ERP. The 
Navy ERP program uses a product from SAP Corporation, the largest provider of ERP 
solutions in the world. 

Navy ERP is the DoN finaneial system of record, meaning that it provides reliable 
information for naval leadership to keep our fleet moving forward. Navy ERP streamlines 
the U.S. Navy’s business operations, namely financial and supply chain management 
(U.S. Navy ERP, 2013). In 2010, Navy ERP Release 1.1 (Single Supply Solution) went 
live on Mareh 17 at NAVSUP, enhaneing the ability for U.S. Navy supply ehain 
managers to effeetively and effieiently provide Sailors and ships with the items that they 
need every day (NAVSUP, 2010). 

2. Impact on the CILS-TAT 

Due to the roll-out of Navy ERP in 2010, all inventory management programs 
were plaeed in a phasing plan to gradually implement the system and mitigate any 
adverse effeets that might oeeur by rushing through the implementation proeess. The 
highest priority was given to the requirements that the fleet needed on a day-to-day basis, 
thereby relegating all re-allowaneing proeesses to a lower priority until the system had 
been proven. This measure was taken to ensure that no high-priority requirements were 
missed during the migration period. The measure led to the suspension of all CIES- 
TATs in 2009 until further notiee. 
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The Navy ERP system contains several improved capabilities that will 
significantly impact the effect of the CILS-TAT in the future. The integrated processes 
within Navy ERP use a single set of data, automatically disseminate information from 
one entry to all parts of the process where it is required, and make the entire end-to-end 
information stream visible to managers with responsibility over the processes (El.S. Navy 
ERP, 2013). There are upgraded supply chain and financial management capabilities that 
provide for better asset visibility across the enterprise as well as better reporting of the 
financial impacts of CILS-TAT implementation. The improved supply chain suite also 
aids in acquiring the parts to fill the newly added allowances. 

As of 2013, Navy ERP has proven to be a significant success across the U.S. 
Navy, and senior leadership within the logistics community is once again ready to discuss 
the return of the CILS-TAT. If CILS-TAT is proven to be a wise investment and re¬ 
implemented, it should benefit greatly from the increased functionality recognized 
through the use of Navy ERP. 
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III. METHODOLGY 


A. INTRODUCTION 

Included in the following chapter is a brief overview of the methodologies or 
models used in conducting the BCA regarding the effectiveness of CILS—TAT. The 
reasoning behind examining the following models was threefold. First we utilized the 
eost-benefit analysis (CBA) to attach a dollar amount necessary to achieve a certain level 
of readiness. Further, a CBA allowed us to determine the cost and benefits to be weighed 
against a performance metric defined by the stakeholder. Next, we developed a 
regression model to evaluate the effeetiveness of CILS—TAT from the before and after 
periods. The regression model was further refined through the use of cluster analysis, 
whieh enabled the grouping of data in an attempt to find commonality experienced by 
varying sets of ships. Finally, we included in this chapter an overview of the knapsack 
model. This information is eontained within this chapter for use in making improvements 
to the eurrent CILS—TAT framework. The aforementioned methods provide a 
comprehensive set of tools for developing a thorough BCA. 

B. BUSINESS CASE ANALYSIS 

I. Background 

In business and throughout the DoD, the use of BCA has gained popularity 
through the leadership of the Offiee of the Secretary of Defense (OSD) to support 
strategy decisions regarding program implementation. A BCA provides a best-value 
analysis that considers not only cost but also other quantifiable and non-quantifiable 
faetors supporting an investment deeision (DAU, 2013). Depending on the type of BCA 
it may be used throughout the life cycle of the projeet. Speeifically, the BCA should be 
used in further deeisions to sustain or enhance the solution and to refine estimation of 
benefits and costs for future projeets in the organization (DAU, 2013). The BCA ean be 
thought of as an expanded CBA with the intent of determining a best-value solution. The 
BCA process goes beyond cost/benefit or traditional eeonomic analyses by doeumenting 
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how each alternative fulfills the strategic objectives of the program and the resulting 
impact on stakeholders (DAU, 2013). 

Within the BCA, a CBA is used as a method to aid decision-makers in 
quantifying differences between projects. It provides a comparative assessment of all 
benefits anticipated and the costs incurred in various iterations. CBAs thereby allow 
decision-makers to pick the optimal solution for the allocation of scarce resources. 

When conducting a CBA, three frames of reference can be used to approach the 
analysis: ex ante, ex post, and in medias res (Boardman, 1996, p. 3). These different 
reference frames define the time at which the analysis is completed. In ex ante analysis, a 
CBA is conducted prior to the start of a project. The advantage of ex ante analysis is that 
it facilitates the decision-maker to ensure better decisions about the allocation of 
resources; however, it does not provide the most complete understanding of the actual 
benefits or cost assumed—there is a high degree of uncertainty. An in medias res 
analysis is completed as the project progresses. While this method provides a reduction 
of uncertainty, it reduces the ability of the decision-maker to have full control of the 
proper allocation of scarce resources. If a project has been completed, an ex post CBA 
can be conducted. This method does not provide a decision-maker with the ability to 
allocate resources, since they have already been expensed, but is the most accurate 
method to understand the actual impacts of benefits and costs. For most companies, ex 
post analysis is not an option because of the large monetary investment in conducting a 
CBA. In general, most companies or government agencies rely on ex ante analysis with 
an understanding that there is an added degree of risk due to inherent uncertainties. 
Looking further into the CBA process, the framework for the analysis consists of nine 
steps (Boardman, 1996, pp. 6-24). The following is a breakdown of each step: 

2. The Steps of a Cost-Benefit Analysis 

The following steps describe a typical CBA: 

1. Decide whose benefits and cost count. In this step, the analysts must 
choose the scope of the analysis and determine the target group. Specifically, the 
analysts must decide if they are looking from the perspective of a guardian. This decision 
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becomes integral in later steps because it will define how costs or benefits are viewed—a 
benefit to a guardian may be a cost to a spender. In general, guardians ignore 
nonfinancial social benefits. For social benefits in the federal government, it is assumed 
that the analysis is being completed from the spender’s (society’s) perspective vice the 
guardian’s (federal government’s) perspective (0MB, 1992). 

2. Select the portfolio of alternative projects. This step allows the analysts to 
bound the project being analyzed. For simplicity, only one project is analyzed at a 
time—in theory, there are infinite numbers of alternatives. This step mitigates 
uncertainty associated with complicated relationships among parameters being 
manipulated. As the number of possible values increases among alternative projects, the 
overall alternatives increase exponentially. As Boardman (1996) observed, “If there were 
n dimensions, each with k possible values, there would be kn alternatives ... there would 
be 27 mutually exclusive alternatives. Neither decision makers nor analysts can 
cognitively handle comparison among such a large number of alternatives” (p. 13). 
Often, external constraints further restrict the project from reaching the optimal output 
levels. The analysts must be cognizant of these factors and what limitations are being 
imposed. 

3. Catalog potential (physical) impacts, and select measurement indicators. 
Within this step, the analysts define variables, both tangible and intangible, that are being 
considered and quantify their impact to the overall project. Chosen variables must ensure 
that there is a cause-and-effect relationship between a tangible outcome and a society. 
When defining variables, analysts must explicitly state all assumptions, especially if 
assumptions are made about future benefits or costs (0MB, 1992). Once the list is 
collected, the variables are aggregated as either a benefit or a cost. 

4. Predict quantitative impacts over the life of the project. This step takes 
each variable impact and projects its value and changes over the life of the project. In 
some instances, the projections are taken past the life of a project if the project is 
anticipated to have a continued impact on alternate projects following termination. This 
section looks to correlate the impact of a variable to a tangible value—for example, lives 
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saved per year or part reduetion per year. If existing projeets are available, historieal data 
ean be used as a baseline to extrapolate the impact of possible changes. 

5. Monetize (attach a dollar value to) all impacts. The goal of this step is to 
associate a monetary amount to all impacts in terms of the dollar amount saved or lost for 
each variable. This goal ensures a common set of units for comparing one variable to 
another. The monetary amount associated with an impact is often valued based on a 
willingness to pay (0MB, 1992). This relationship can be extrapolated, for a well- 
defined product, from market prices. Variables such as parts reduction or labor reduction 
will be monetized based on the market price for labor or the price of each part reduced. 

6. Discount for time to find present values. If a project is expected to have 
benefits or costs realized over the course of the project’s life (years), these future values 
must be aggregated in their present value (PV) for comparison. The PV of a future cost 
or benefit can be calculated using the PV equation (see Figure 6). 


I uni;v V.iluiMt: Hi rK:.: ortitts: 
it - df 

Figure 6. Present Value Equation 
(From Brealey, Myers, & Allen, 2011, p. 104) 

In Figure 6, d represents the opportunity cost of capital and t represents the 
number of years in the future that the benefit or cost is realized (Brealey et ah, 2011, p. 
104). The opportunity cost of capital represents the expected return not realized because 
of a project investment compared to financial securities (Brealey et ah, 2011, pp. G-11). 

7. Sum: Add up the benefits and costs. Using the PV of each variable, the 
analysts sum up all benefits and costs to calculate the net present value (NPV) of each 
alternative (see Figure 7). 
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Figure 7. Net Present Value Equation 
(Erom Brealey et ah, 2011, p. 104) 
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8. Perform sensitivity analysis. A sensitivity analysis allows the analysts to 
eapture the impact of uncertainty for each variable within a project. This analysis will 
identify to a decision-maker what variables can absorb higher degrees of uncertainty with 
marginal impacts to the outcome. It is very rare that the impact of a variable or its 
valuation per unit impact is known completely. A sensitivity analysis can be completed 
by either manually adjusting variables to see their impact to the output or by conducting a 
model analysis (Ragsdale, 2008, p. 136). 

9. Recommend the alternative with the largest NPV. When choosing 
between alternatives, the decision-maker should choose the project or alternative with the 
highest NPV. If the decision is between all negative NPVs, the project or alternative with 
the lowest negative value should be chosen. After the analysis, there may be an instance 
where doing the status quo will result in a higher NPV—sometimes doing nothing 
different is a better option. 

C. KNAPSACK PROBLEM 

1. Introduction 

Integer and combinatorial optimization deals with the problems of maximizing or 
minimizing a function of many variables subject to (a) inequality and equality constraints 
and (b) integrality restrictions on some or all of the variables. Because of the robustness 
of the general model, a remarkably rich variety of problems can be represented by 
discrete optimization models (Nemhauser & Wolsey, 1988). 

The knapsack problem (KP) is founded in the scope of integer and combinatorial 
optimization. Suppose that a hitchhiker has to fill up his knapsack by selecting from 
among various possible objects those which will give him maximum comfort. This very 
rudimentary question formulates the basis of the KP. According to Martello and Toth 
(1990), the KP can be mathematically formulated by numbering the objects from 1 to n 
and introducing a vector of binary variables Xj (j — 1 , ...,71) with the following 
assignments: 

^ _f 1 if object j is selected; 

^ to otherwise. 
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Then, if pj is a measure of the eomfort given by objeet j,Wj, its size, and c (the size of 
the knapsaek), our problem will be to seleet, from among all binary veetors x satisfying 
the eonstraint 

(3) 

the one whieh maximizes the objeetive funetion 

PjXj. (4) 

The KP has attraeted and been intensively studied by both theorists and 
praetitioners. Nemhauser and Wolsey (1988) espoused that an important and widespread 
area of applieation eoneerns the management and effieient use of searee resourees to 
inerease produetivity. These applieations inelude operational problems sueh as the 
distribution of goods, produetion seheduling, and maehine sequeneing. They also inelude 
planning problems sueh as eapital budgeting, faeility loeation, and portfolio analysis. 

For the purposes of this researeh, we address a eapital budgeting model with the 
problem of seleeting among various allowaneing possibilities so as to maximize the total 
operational readiness without exeeeding the available funds. Aeeording to Christofides, 
Mingozzi, Sandi, and Toth (1979), this model can be directly expressed as a zero-one (0- 
1) KP (each allowance possibility is either accepted or rejected). 

2. Zero-One Knapsack Problem 

The 0-1 KP is the most important KP and one of the most intensively studied 
discrete programming problems. According to Martello and Toth (1990), the 0-1, or 
binary, KP is given a set of n items and the knapsack, with 

Pj — profit of item j, 

wj = weight of item j, 

c = capacity of the knapsack. 

Select a subset of the items so as to 

maximize Z = PjXj (5) 
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subject to 


EjLiWjXj < c 


(6) 


Xy = 0 or 1 j E N = {1,..., n], 

where 


X; 


1 if item j is selected; 
0 otherwise. 


The reason for such an interest basically derives from three facts: (a) the 0-1 KP can be 
viewed as the simplest integer linear programming problem, (b) it appears as a 
subproblem in many more complex problems, and (c) it may represent a great many 
practical situations (Martello & Toth, 1990). 


3. Bounded and Unbounded Knapsack Problem 

One common variant of the 0-1 KP model is that each item can be chosen 
multiple times. The bounded knapsack problem (BKP) specifies that for each item j, an 
upper bound bj (which may be a positive integer, or infinity) is on the number of times 
that item j can be selected (Martello & Toth, 1990). The BKP equation, as defined by 
Martello and Toth (1990), asserts to 


maximize 


(V) 

subject to 

WjXj < c, 

(8) 


0 < Xj < bj and integer,) G iV = (1, ...,n}. 


The BKP is 

a generalization of the 0-1 KP, in which bj — 1 for all j G N. 

We assume. 

without loss of generality, that 



Pj,Wj,bj and c are positive integers, 



bj Wj > c, 

(9) 


bjWj < c for j G N. 

(10) 


The unbounded knapsack problem (UKP), sometimes referred to as the integer 
KP, does not put any upper bounds on the number of times that an item may be selected 
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(Christofides et al., 1979). This type of scenario might be applicable to the allowancing 
product in that the allowancing model determines that multiple assets of the same line 
item are necessary to support the demand signal of the ship. According to Martello and 
Toth (1990), the model seeks to 


maximize 

z = Slfl PjXj 

(11) 

subject to 

sr=iWiXi < c, 

(12) 


Xj > 0 and integer, j G n = (1,..., n). 

Through the use of combinatorial optimization, any possible revision of the 
current CILS-TAT model would be streamlined to ensure the greatest yield for the 
financial investment. The application of the KP model to this project is vitally necessary 
to maximize the effectiveness of the allowance products developed for U.S. Navy ships. 

D. REGRESSION ANALYSIS 

I. Overview 

Taken from a very broad vantage point, regression analysis can be understood as a 
statistical tool for the estimation of relationships between variables. It includes many 
techniques for modeling and analyzing several variables when the focus is on the specific 
relationship between a dependent variable and one or more independent variables. More 
specifically, regression analysis helps to explain how the typical value of the dependent 
variable changes when any one of the independent variables is varied while the other 
independent variables are held fixed (Lind & Mason, 1993). Regression analysis can 
have varying uses, ranging from prediction (to include the forecasting of time-series 
data), inference, hypothesis testing, and modeling of causal relationships (Anderson, 
Sweeney, & Williams, 2000). 

Linear regression was the first type of regression analysis to be studied rigorously 
and to be used extensively in practical applications. According to Anderson et al. (2000), 
the equation includes the effects or regression coefficients (P), a dependent variable (y), 
independent variables (x), and the error term or noise (s). A regression model relates the 
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dependent variable y to a funetion of x and P, written y = /(x, The regression 
analysis model can have a single independent variable or multiple independent variables 
as seen in Figure 8. 


y = Pix-\- £ 

Figure 8. Linear Regression Analysis Equation 
(From Lind & Mason, 1993) 

With the development of a regression analysis, the researcher must develop a set 
of assumptions regarding the probability distribution of the errors that must hold in order 
for the model to be effective. Statistical tests are then made on the basis of these 
assumptions (Lind & Mason, 1993). 

It is worth mentioning here a couple of key assumptions that must be considered 
when evaluating the output from the model. The first assumption that must be addressed 
is the presence of multicollinearity, which is defined as the problem where changes in 
two variables are nevertheless highly correlated—to the point that it is difficult to 
separate their effects on the dependent variable (Sykes, 1993). The other assumption that 
must be addressed is the presence of normality. The evaluation of normality is done by 
calculating the random error in the relationship between the independent variables and 
the dependent variable in a regression model; the random error should be normally 
distributed. 

Once the regression model is developed, the researcher must assess the statistical 
significance of the estimated relationship to determine the strength of the relationship 
predicted by the data against the true relationship. Along with determining the statistical 
significance, the investigator must determine the goodness of fit for the model by 
evaluating the R value. The R value is a number between 0 and 1 that describes how 
well a regression fits a data set. Once the model is developed, assumptions have been 
validated, and the statistical significance and goodness of fit has been proven, the model 
is ready to be used for various applications (Anderson et ah, 2000). 
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2. Linear Regression 

When developing a simple linear regression, the researcher must determine which 
variables of interest must be taken from the data set to satisfy the question posed by the 
hypotheses. This data will then be graphically depicted using a scatter plot diagram. The 
resulting display will give the researcher the ability to easily determine the general 
correlation between the two variables when the other covariates are held fixed. 

The regression is further defined through the use of the least squares method to fit 
a line to the distribution. We call the estimate of the line’s intercept bo and that of the 
line’s slope bi. The estimated or predicted value of y is denoted as y. Within the scatter 
plot, the hypothesized relationship thus implies that somewhere on the diagram may be 
found a line with the equation y = bo + b-^x. In the least squares method, we minimize 
the sum of squared differences between y and y. Then we define a residual for 
observation i to be ei — yi — y^ and minimize the sum of squared errors, Td=i 
(Anderson et al., 2000). 

The same fundamental equation can also be applied when multiple independent 
variables are introduced. The technique is called multiple regression, and it allows 
additional factors to enter the analysis separately so that the effect of each can be 
estimated. The technique is valuable for quantifying the impact of various simultaneous 
influences upon a single independent variable (Sykes, 1993). Furthermore, because of 
omitted variables bias with simple regression, multiple regression is often essential, even 
when the researcher is only interested in the effects of one of the independent variables 
(Sykes, 1993). The general form of the model is y — Pq + p^x^ + P 2 X 2 + —h PpXp + 
s, and the estimated relationship is y = hg + + 62 X 2 + —\- bpXp. We use the least 

squares method to find the values of bo, bi... bp that minimizes the sum of the squared 
differences between yj — fy (Lind & Mason, 1993). 

3. Assumptions 

As with any model, there are fundamental assumptions that must be understood 
before the researcher can proceed with the interpretation of the results. There are six 

important assumptions centered on regression analysis. 
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1. Relationship possesses linearity. 

2. Error terms (s) are normally distributed. 

3. At every x value, the error terms have eonstant variance 
(homoscedasticity). 

4. The error terms are independent of each other. 

5. There is a lack of multicollinearity. 

For the regression to possess linearity, the mean of the response variable is a 
linear combination of the parameters (regression coefficients) and the predictor variables. 
Because the predictor variables are treated as fixed values, linearity is really only a 
restriction on the parameters (Noether, 1971). In order to check for model linearity and 
constant variance, a scatter plot would be used. This is done by plotting the residuals 
against the predicted values. The researcher would hope to see a plot that reveals no 
patterns; in other words, he or she wants the plot to have a lot of randomly distributed 
points. If the error terms have different variances, the researcher would see the spread in 
the residuals changing as a function of the predicted value (Anderson et ah, 2000). 

To test the assumption that error terms are normally distributed, the researcher 
must evaluate the residuals. This process is conducted by constructing a histogram of the 
residuals. If the distribution looks bell-shaped, the researcher can feel comfortable that 
the error terms are close to normally distributed (Anderson et ah, 2000). 

The final assumption in linear regression is that the independent variables are 
truly independent of each other. The violation of this assumption is referred to as 
multicollinearity. If the researcher is only interested in prediction, then multicollinearity 
may not represent a large problem. If the researcher is trying to explain the relationships 
between dependent and independent variables, it does cause problems (Lind & Mason, 
1993). 

The main problem is that the standard error of the regression coefficients is highly 
inflated; hence, the estimated regression coefficients have large sampling variability. 
Estimated regression coefficients tend to vary widely from one sample to the next when 
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the independent variables are highly eorrelated. Another problem is the interpretation of 
the estimated eoefficients. When the explanatory variables are eorrelated, the researeher 
eannot change one variable without the correlated variable(s) changing at the same time. 

There are two ways of identifying multicollinearity within the model, and the 
approach varies depending on preference. The researcher can either look for the effects 
of the correlation or focus on the causes. One example of looking for effects might be 
identifying large changes in the estimated regression coefficients when a variable is 
added or deleted. An example of looking for causes would be to recognize large 
correlation coefficients between independent variables in the correlation matrix (Neter, 
Wasserman, & Kutner, 1990). 

4. Statistical Inference and Goodness of Fit 

The process of determining the statistical inference of the model is done by 
looking at the goodness of fit of the model and whether or not the overall relationship is 
significant. In other words, is the dependent variable related to any of the independent 
variables? Variables are evaluated by testing a hypothesis of Ho and Ha. The hypothesis 
is tested by comparing the amount of variation explained by the independent variables to 
the amount of variation left unexplained. The unexplained variance is the residual mean 
square. The explained portion is referred to as the regression mean square, and the F 
statistic is the ratio of explained to unexplained variance (Anderson et ah, 2000). 

All parameters were analyzed using the Welch Two-Sample Mest. This test 
method assumes unequal variance between data sets and that the two data sets are not 
paired. Under the assumption of unequal variances, the denominator of the t-test is not a 
function of the pooled variance, as would be the case in the Student’s t-test (Keller, 
2009). Equations 13 and 14 give the equation for the Welch t-statistic. 


X1-A2 
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V = 


rs/^s/V 
U, nJ 


s: 


N,\N,-\) N,\N,-\) 


(14) 


This test was completed using the t-test function in the stats package of R Studio 
software. Each data set was evaluated using three different hypotheses tests: 


Two Sided \ 


^0 • Before After ^ 
f^Before- After 


Less { 


^0 Before After ^ 
Before ~ After ^ ^ 


Greater { 


^0 Before MAfter ^ 

^1 Before ~ After ^ ^ 


(15) 


(16) 


(17) 


Using these three tests ensures that any movement, positive or negative, in the 
data set means is fully captured in the analysis. In each case, a 95% confidence interval 
(Type I error < 0.05) was used as a baseline metric to establish statistically significant 
changes in the data sets. 


E. CLUSTER ANALYSIS 
1. Overview 

Cluster analysis is a subset of data analysis tools called data mining. This 
analysis seeks, by grouping information, to find the interrelationships across various 
parameters or variables to understand the structure of data sets. Cluster analysis places 
data of similar values together to create a series of N homogeneous groupings (Ye, 2003). 
For the purposes of this project, in an effort to understand why certain ships experienced 
a statistically significant change in allowance effectiveness, cluster analysis is used to 
find the relationship among various source codes and allowance effectiveness. For 

example, for the ships that achieved a statistically significant change in allowance 
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effectiveness, was it the respective change in source code A or source code G which 
caused the overall change in allowance effectiveness? What are the differentiating factors 
among the ships? Cluster analysis will show the thread among parameters which causes 
a ship to see statistically significant change in allowance effectiveness. 

2. K-means Algorithm 

K-means falls under the partitional method of cluster analysis. In this analysis, 
data is split into user-defined K clusters. The correct number of clusters is a subjective 
value, depending on constraints within the application or oftentimes found by iterating 
through various possibilities to find the best match (Ye, 2003). The overall goal is to find 
a K value that partitions the data into rational groupings with minimum overlap among 
clusters and no empty clusters. 

The k-means algorithm uses a five-step process, seen in Figure 9, to identify the 
location and size of the clusters. In the first step, K number of cluster seed points are 
identified from the data set and represent an initial estimate of the location of the center 
of the clusters (initial centroids). 


Basic K-means algorithm 


1: Select K points as initial centroids. 

2: repeat 

3:From K cluster by assigning each point to its closest centroid. 

4:Recompute the centroid of each cluster. 

5: until Centroids do not change 

Figure 9. Cluster Analysis—The Basic K-means Algorithm 

(From Ye, 2003) 

Once the location of the clusters has been identified, each of the data points is 
then associated with a respective cluster based on its proximity to the cluster centers. 
Fundamentally, the proximity calculation is an optimization problem whose objective 
function is to minimize the sum of the squared error (SSE), Equation 18, based on the 
Euclidian distance of each point to the cluster center (Ye, 2003). 


42 










K 

minimize SSE = ^'^dist[c.,x] (18) 

/-I xeC, 

Turning to geometry, the Euelidian distanee represents nothing more than the shortest 
distanee between two points. In n-dimensional spaee, this distance is equal to Equation 
19, where Ci and Xi represent the center point of the cluster and x represents a point in the 
data set: 


n 

V J=1 


(19) 


Once the respective values have been assigned to one of the clusters, a new centroid is 
calculated based on the mean of the values within the cluster. This process is repeated 
until there is no change in the location of the centroids. 

The results of the K-means analysis are a series K of centroid points for each 
cluster along with standard deviations of the ellipses representing a specified confidence 
interval. All cluster analyses in this project were completed using the k-means function 
in R software and the k-means cluster analysis function in JMP statistical discovery 
software from SAS using a Type I error of less than 0.05. 


F. CONCLUSION 

This chapter has outlined the various analytical approaches taken to analyze our 
data set in the context of a BCA. In reviewing this chapter, the reader should have 
confidence in the level of analysis and research undertaken. The content should also 
expand the reader’s knowledge of the techniques used. Evaluating the impacts of CIES— 
TAT on individual ships’ performance is a multivariate problem requiring multiple 
approaches. The data mining methods discussed provide a methodical approach to 
analyzing the numerous variables that have a potential impact on ship performance and, 
where possible, isolate CIES—TAT effects. These techniques are widely accepted as 
relevant approaches when conducting a BCA, and the authors felt that the inclusion of 
each was both appropriate and relevant to the research questions. Each of the models 
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selected provides a different angle from which the data can be evaluated, and the use of 
multiple approaches ensures accuracy and reliability of the BCA. 
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IV. DATA ANALYSIS 


A. DATA RETRIEVAL 

The data used during the eourse of this project was provided entirely by NSLC. 
The project required a significant amount of historical data for each of the 18 ships we 
analyzed and included a summary of all CASREPs submitted from 2003-2011, in-port 
and underway steaming hours, 3M registered demand data from 2003-2011 covering 
more than 500,000 material requests, monthly percentages that ships were free from 
Category 3 and Category 4 CASREPs, and supply effectiveness rates. 

NSLC also provided a detailed report for each of the 18 CILS-TATs conducted 
during our period of analysis. These reports contained the number of items 
recommended to be carried as allowances, the cost of each item identified, and the 
mission criticality code. 

As we received data, we grouped and normalized it to verify continuity of 
information and determine whether gaps existed in the analysis window. Of the most 
critical data elements, the one with the largest amount of missing data was the CASREP 
summary information. The project accounts for a total of 864 months of operational data 
from the 18 ships we surveyed. From those 864 months, there were 42 months in which 
the CASREP data was either missing or incomplete and could not be provided by NSLC. 
Since these 42 months account for exactly 4.86% of our total sample, the amount of data 
provided was deemed sufficient for further analysis. 

Due to the large amount of supporting data available, in our analysis, we were 
able to control for a number of variables that would otherwise bring the relevancy of our 
conclusions into question. One variable that we were unable to account for, however, 
was the amount of funding provided to the ships for traditional comprehensive 
allowancing. Since funding provided in this format is assumed to vary from ship to ship, 
the robustness of a ship’s allowance product will impact both its supply effectiveness 
rates and its incidence of CASREPs. 
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For the purposes of this projeet, the ships and speeifie time periods we analyzed 
are listed in Table 3. The ships represent both the Atlantie and Paeifie fleets and are the 
eurrent members of the U.S. Navy’s BMD fleet. 


Table 3. List of Ships and CILS—TAT Periods Analyzed 


Name 

UIC 

Date Range 

MONTEREY CG61 

21450 

11 1/2003 - 11/30/2007 

SHILOH CG67 

21657 

10/1/2004- 10/31/2008 

LAKE ERIE CG70 

21827 

2 1 2005 - 2 28/2009 

VELLA GULF CG 72 

21829 

7/1/2003 - 7/31/2007 

PORT ROYAL CG73 

21830 

9/1/2003 - 9 30/2007 

JOHN PAUL JONES DDG53 

21313 

8/1/2004- 8/31/2008 

STOUT DDG55 

21685 

12/1/2004- 12 31/2008 

JOHNS MCCAIN DDG 56 

21686 

5/1 2005 - 5/31/2009 

RUSSELL DDG 59 

21821 

6/1/2003 - 6 30/2007 

PAUL HAMILTON DDG 60 

21822 

10/1 2004- 10 31/2008 

RAMAGE DDG 61 

21823 

7/1/2003 - 7 31 2007 

FITZGERALD DDG 62 

21824 

6/1 2004 - 6/30/2008 

BENFOLD DDG 65 

21940 

10/1/2004- 10/31 2008 

THE SULLIVANS DDG 68 

21942 

11/1/2003 - 11/30/2007 

MILIUS DDG 69 

21943 

3/1/2005 - 3/31/2009 

HOPPER DDG 70 

21944 

11/1 2003 - 11/30/2007 

DECATUR DDG 73 

21947 

8/1/2003 - 8/31 2007 

HIGGINS DDG 76 

21950 

5/1 2003 - 5/31/2007 


B. CILS-TAT ANALYSIS WINDOWS 

Throughout the analysis, all ships were analyzed individually. The overall data 
set eovers periods from December 2003 to January 2011. It is worth noting that for the 
remainder of this analysis, a period represents a month of time unless otherwise specified. 
For parameters that are sampled at frequencies greater then monthly, the data has been 
averaged or summed as necessary to ensure consistent analysis periods. 
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For the various ships, CILS—TAT was implemented at different times during the 
periods from June 2003 to May 2005. When analyzing the individual ships’ parameters, 
the data sets are broken up between the 24 periods before the implementation of CILS— 
TAT and the 24 periods after. In total, the data set eovers 48 periods. Table 4 identifies 
the applicable date ranges for the CILS—TAT data set windows. The Before column in 
Table 4 indicates the month and year that starts the 24 months before CILS—TAT, and 
similarly, the End column indicates the final month and year of the 24 months following. 
The date of CILS—TAT implantation is indicated by the Entered column. When grouping 
individual ships before and after CIES—TAT, respective periods (0-48) are combined 
accordingly, ensuring consistency in period-to-period analysis. 


Table 4. CILS—TAT Analysis Windows 


Ship Name 

Before 

Entered 

End 

USS Monterey 

Nov/2003 

Nov/2005 

Nov/2007 

USS Shiloh 

Oct/2004 

Oct/2006 

Oct/2008 

USS Lake Erie 

Feb/2005 

Feb/2007 

Feb/2009 

USS Vella Gulf 

Jul/2003 

JuF2005 

Jul/2007 

USS Port Royal 

Sep/2003 

Sep/2005 

Sep/2007 

USS John Paul Jones 

Aug/2004 

Aug/2006 

Aug/2008 

USS Stout 

Dec/2004 

Dec/2006 

Dec/2008 

USS John S McCain 

May/2005 

May/2007 

May/2009 

USS Russell 

Jun/2003 

Jun/2005 

Jun/2007 

USS Paul Hamilton 

Oct/2004 

Oct/2006 

Oct/2008 

USS Ramage 

Jul/2003 

JuF2005 

Jul/2007 

USS Fitzgerald 

Jun/2004 

Jun/2006 

Jun/2008 

USS Benfold 

Oct/2004 

Oct/2006 

Oct/2008 

USS The Sullivans 

Nov/2003 

Nov/2005 

Nov/2007 

USS Milius 

Mar/2005 

Mar/2007 

Mar/2009 

USS Hopper 

Nov/2003 

Nov/2005 

Nov/2007 

USS Decatur 

Aug/2003 

Aug/2005 

Aug/2007 

USS Higgins 

May/2003 

May/2005 

May/2007 
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C. MODEL OUTPUT AND ANALYSIS 

1. Allowance Effectiveness Results 

The analysis of CILS—TAT’s impact on allowance effectiveness starts with 
looking at the descriptive statistics for the before and after data sets. Throughout this 
evaluation, the expected result is that the mean of the before data set is less than the mean 
of the after data set. This is consistent with Hypothesis 2 (Equation 16) noted in the 
Methodology chapter under the Regression Analysis section. Looking at Table 5, which 
outlines baseline statistics for the before and after data sets, the entire before data set has 
an average allowance effectiveness of 66.03%, with values ranging from 100% to 
12.50%. Between the CG and DDG groups, the average allowance effectiveness is 
70.08% and 64.47%, respectively. While these values appear to be dissimilar, they are 
well within one standard deviation of each other and the overall average, thereby making 
them statistically similar. For purposes of comparison, the data shared between the two 
classes of ships are similar from an allowance effectiveness perspective. 

For the after data, the overall averages range from 25.30% to 98.40%, with an 
average of 71.12%. Similarly, CG and DDG groups average 73.97% and 70.02%, 
respectively. The absolute change in means range from -17.57% to -0.25%. Overall, the 
average change in means for the data sets was -5.09. Categorically, CGs experienced an 
average change of -3.90, compared to an average change of -5.56 experienced by DDGs. 
To ensure consistency in the Welch Two-Sample t-test, the change in mean—as 
calculated in Table 5—is the difference between the mean of the before data set minus 
the mean of the after data set. Therefore, a negative number in the Change in Mean 
column from Table 5 equates to an increase in allowance effectiveness from the before 
period to the after period. 


48 



Table 5. Summary of Changes in Means due to CILS—TAT 


Summary Statistics: Allowance EfTectiveness (CILS-TAT] 

Before (2 years) After [2 Years] 


Max Min Averse StdDev Variance Max Min Average StdDev Variance 



CG 

61 

85.20 

47.10 

67.87 

11.89 

141.48 

97.60 

59.00 

80.23 

9.15 

83.78 

-12.37 


CG 

67 

93.80 

46.30 

69.71 

12.65 

160.11 

91.50 

45.90 

73.79 

10.49 

110.07 

-4.08 

CG 

CG 

70 

88.30 

56.70 

69.34 

7.67 

58.86 

79.50 

41.10 

65.55 

11.03 

121.75 

3.80 


CG 

72 

86.20 

59.20 

73.80 

7.12 

50.65 

91.70 

25.30 

75.88 

13.79 

190.13 

-2.08 


CG 

73 

83.80 

46.80 

69.67 

9.88 

97.56 

93.60 

48.60 

74.42 

10.96 

120.22 

-4.75 

CG Overall (Avgl 

93.80 

46.30 

70.08 

10.11 

102.27 

97.60 

25.30 

73.97 

12.00 

143.99 

-3.90 


DDG 

53 

83.60 

16.90 

56.91 

15.42 

237.75 

90.70 

44.70 

71.01 

15.11 

228.29 

-14.10 


DDG 

55 

100.00 

39.70 

68.09 

14.50 

210.29 

93.40 

46.60 

66.32 

12.57 

157.94 

1.78 


DDG 

56 

86.20 

12.50 

63.70 

19.38 

375.76 

89.60 

41.10 

71.64 

12.48 

155.71 

. 7.93 

All Ships 

DDG 

59 

86.00 

37.80 

62.39 

13.71 

188.05 

85.10 

53.30 

71.87 

10.31 

106.24 

-9.48 

DDG 

60 

79.60 

40.80 

62.45 

11.02 

121.43 

89.00 

28.70 

67.10 

15.66 

245.36 

-4.65 


DDG 

61 

86.40 

43.90 

64.71 

10.27 

105.50 

95.70 

29.00 

67.52 

14.50 

210.18 

-2.81 

DDG 

DDG 

62 

98.70 

38.70 

63.28 

17.58 

309.22 

92.50 

46.00 

67.13 

12.95 

167.82 

-3.86 


DDG 

65 

89.10 

15.80 

63.65 

15.23 

231.86 

92.40 

38.40 

73.69 

11.45 

131.01 

-10.04 


DDG 

68 

93.10 

47.20 

67.27 

11.40 

130.07 

82.70 

48.40 

64.30 

10.00 

100.01 

2.97 


DDG 

69 

91.50 

31.60 

66.68 

12.93 

167.24 

98.40 

69.50 

84.26 

7.96 

63.38 

-17.57 


DDG 

70 

98.90 

50.50 

72.57 

15.18 

230.42 

91.20 

43.40 

72.83 

13.17 

173.53 

-0.25 


DDG 

73 

89.30 

43.30 

67.57 

10.43 

108.88 

92.30 

45.10 

69.10 

11.99 

143.76 

-1.53 


DDG 

76 

78.40 

31.70 

58.79 

11.79 

139.07 

92.00 

29.00 

63.53 

16.04 

257.41 

-4.73 

DDG Overall )Avg] 

100.00 

12.50 

64.47 

14.30 

204.55 

98.40 

28.70 

70.02 

13.60 

184.83 

-5.56 

AIISIhMCKeralMAvl 

100.00 

12.50 

66.03 

13.50 

182.17 

98.40 

25.30 

71.12 

13.28 

176.26 

-5.09 


While all 18 ships experieneed a ehange in the means of their data sets, only six 
ships, highlighted in yellow, had a change based on the Welch Two-Sample t-test, which 
was statistically significant. For the remaining ships that did not meet the threshold for a 
Type I error of less than 0.05, there was not enough evidence to prove that the variance in 
the data sets and the respective changes in means was not purely based on chance. The 
intent of this analysis is to show that for six of the ships in the sample, CILS—TAT was 
directly responsible for the change in effectiveness. The results of the Welch Two- 
Sample t-test are summarized in Table 6. 
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Table 6. Summary of Allowance Effectiveness and the Welch Two-Sided t-Test 


T-Tot: AUoi* 


Mean of 
Before 

Mean of 
After 

Lcm Alternative - 

TSUtlilfc MWue DOF LCL 

Mean 

UCL 


CG6I 

67.87 

80.23 

-12.37 

-4.04 

0.000109 

43.17 

- oc 

-7.22 


CC3 67 

69.71 

73.79 

-4.08 

-1.22 

0.115242 

44.47 

- oc 

1.56 


CG 70 

69.34 

65.55 

3.80 

1.38 

0.913032 

41.03 

- oc 

8.41 


CG 72 

73.80 

75.88 

-2.08 

-0.66 

0.258376 

34.44 

oc 

3.28 


CG 73 

69.67 

74.42 

-4.75 

-1.58 

0.060877 

45.51 

- oc 

0.31 


DIXi 53 

56.91 

71.01 

-14.10 

-3.20 

0.001247 

45.98 

^ oc 

-6.70 


DDG 55 

68.09 

66.32 

1.78 

0.45 

0.673693 

45.09 

- oc 

8.35 


DDG 56 

63.70 

71.64 

-7.93 

-1.69 

0.049876 

39.27 

- oc 

-0.01 

Individual 

DDG 59 

62.39 

71.87 

-9.48 

-2.71 

0.004861 

42.70 

- oc 

-3.59 


DIXi 60 

62.45 

67.10 

-4.65 

-1.19 

0.120735 

41.29 

oc 

1.93 


DDG 61 

64.71 

67.52 

-2.81 

-0.78 

0.221230 

41.44 

- oc 

3.29 


DDG 62 

63.28 

67.13 

-3.86 

-0.87 

0.195844 

42.28 

oc 

3.64 


DDG 65 

63.65 

73.69 

-10.04 

-2.58 

0.006679 

42.70 

- oc 

-3.50 


DIXi 68 

67.27 

64.30 

2.97 

0.96 

0.828462 

45.23 

- oc 

8.17 


DIXi 69 

66.68 

84.26 

-17.57 

-5.67 

0.000001 

38.24 

- oc 

-12.35 


DDG 70 

72.57 

72.83 

-0.25 

-0.06 

0.475437 

45.11 

- oc 

6.64 


DDG 73 

67.57 

69.10 

-1.53 

-0.47 

0.319392 

45.14 

oc 

3.92 


DDG 76 

58.79 

63.53 

-4.73 

-1.16 

0.125367 

42;24 

- oc 

2.10 


The primary driver that differentiated statistically significant ships from those that 
were not statistically significant was a large change in the means of the data sets coupled 
with a decrease in the variance and standard deviation of the data from the before period 
to the after period. Figure 10, in the top graph, shows a boxplot for each ship’s allowance 
effectiveness before (blue) and after (red) CILS—TAT. For each ship, the dot in the 
middle is the mean effectiveness, and the block represents the range of +/- one standard 
deviation from the mean. Finally, the whiskers show the difference between the 
maximum and minimum allowance effectiveness. The blue line between the two data 
sets represents the change, direction and magnitude, of the before and after data. 
Similarly, plotted in the bottom section of Figure 10 are the standard deviations before 
and after for each ship. This graph both identifies the change in standard deviation for an 
individual ship and shows these standard deviations in the context of the overall sample 
population. 
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Variance in Allowance Effectiveness Before to After CILS-TAT 
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Figure 10. Variance in Allowance Effectiveness Before and After CILS—TAT 


For five out of the six ships, this is the exact relationship observed: the allowance 
effectiveness increased in all ships where the change was statistically significant. 
However, for DDG 53, the standard deviation of the data sets only changed marginally. 
For this ship, it was the -14.10% change in allowance effectiveness that was well above 
the upper control limit (UCL) of -6.70 that allowed this value to be statistically 
significant. The key is that the ships that experienced a statistically significant change 
saw less variance in allowance effectiveness following the implementation of CILS- 
TAT. The reduction in variance from before to after indicates that CILS—TAT enabled 
the ships to more tightly and accurately control allowance effectiveness. Similarly, 
increased control in variance resulted in a significant increase in the allowance 
effectiveness values post CILS-TAT—the ships were able to maintain a higher value of 
allowance effectiveness. Lrom a manager’s perspective, we can conclude that because of 
CILS-TAT implementation, we noticed an improved level of allowance effectiveness as 
well as a more consistent percentage from before to after. 
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Given the final results of the before and after data, CILS—TAT had a direct and 
positive impact on allowance effectiveness for six out of 18 ships. Of the 12 remaining 
ships, nine saw a positive change in allowance effectiveness, although this change is not 
statistically significant and thus cannot be attributed to CILS—TAT. 

2. CASREP Frequency and POTF Results 

The CASREP data used represents the daily count for Categories 2, 3, and 4. For 
review, CASREPs reflect broken or inoperable equipment onboard U.S. Navy ships. In 
order to aggregate the data, the CASREPs were totaled over the monthly period to be 
consistent with the periodicity of allowance effectiveness data. However, for the 
purposes of CASREP frequency analysis, the various categories of CASREPs were 
summed together with C3 and C4 CASREPs representing less than 20% of all CASREPs. 

Looking at Table 7, seven out of the 18 ships resulted in a positive change in 
mean (before minus after). Of these seven ships, only two of the ships experienced a 
change with a Type I error less than 0.05. None of the ships that experienced a 
statistically significant change in allowance effectiveness saw a similar result for 
CASREPs. Overall, the average change in total CASREPs was -0.5. 
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Table 7. ?-Test Results for Total CASREPs 


luiBtAlri CASREP [OLS-TATI 

KdTNambcr 

Mcaa or 
Before 

Mtmot 

After 

r>Mpl« 

Mean 

TSuditlc 

DOF 

PValna 

ua. 

ucx 

CG61 

10.52 

8.43 

2.09 

0.98 

40.54 

0.166151 

-1.49 

1 aO 

DDG 53 

6.24 

7.82 

-1.58 

-0.82 

28.33 

0.790945 

-4.86 

1 ao 

.4Uo»»ncc pjjj. jg 

EfiecUveocss 

3.83 

6.95 

-3.12 

-1.93 

25.83 

0.967970 

-5.87 

I 90 

P Value <0.05 UIXJ59 

11.41 

5.00 

6.41 

1.69 

17.27 

0.054497 

-0.18 

I ao 

DDG 65 

6.00 

5.95 

0.05 

0.03 

39.64 

0.488102 

-2.76 

♦ 30 

DDG 69 

3.89 

4.20 

-0.31 

-0.25 

34.78 

0.597668 

-2.42 

I ac 

P Value < 0.05 I Avg] 

6.98 

6J9 

0.59 

-0.05 

31.06 

0.510889 

-2.93 

♦ 30 

CG 67 

6.58 

7.30 

-0.73 

-0.37 

39.24 

0.643317 

-4.03 

t 00 

CG70 

5.68 

8.17 

-2.49 

-1.26 

36.14 

0.891559 

-5.83 

I 30 

CG72 

9.68 

11.13 

-1.44 

-0.53 

42.64 

0.700913 

-6.01 

I 90 

CG73 

7.95 

12.21 

-4.26 

-1.61 

39.14 

0.941882 

-8.72 

1 00 

DDG 55 

10.85 

9.32 

1.53 

0.61 

32.28 

0.272666 

-2.71 

1 oo 

.\lloMance „ 

EITccUvencjs rMv^i 

8.95 

14.96 

-6.01 

-1.71 

30.20 

0.951378 

-11.96 

f oo 

P Value >0.05 DDG 61 

8.35 

6.32 

2.03 

0.88 

40.09 

0.191224 

-1.84 

I oo 

DDG 62 

4.68 

6.61 

-1.92 

-1.78 

39.97 

0.958962 

-3.74 

f ao 

DDG 68 

8.05 

11.57 

-3.52 

-1.86 

38.72 

0.964697 

-6.70 

+ 00 

DDG 70 

3.76 

6.79 

-3.03 

-1.32 

29.72 

0.901880 

-6.91 

i ao 

DDG 73 

4.65 

3.36 

1.29 

1.55 

39.87 

0.065117 

-0.12 

+ ao 

DDG 76 

9.25 

4.36 

4.89 

2.23 

30.90 

0.016559 

1.17 

4 30 

P Value > 0.05 (Avg) 

7.37 

8.51 

-1.14 

-0.43 

36.58 

0.625013 

-4.78 

4 X 

AU Ships (Avgi 

7.24 

7.80 

-0.56 

-0-30 

34.74 

0.586972 

-4,17 

4 X 


Turning to the POTF metric, which is calculated as a quarterly percentage, 
represents the amount of time a ship operates without a Category 3 or 4 CASREP. Of the 
18 ships evaluated, seven had a negative change in POTF. Using Table 8, the mean of 
the After data is greater than the mean of the before data with a maximum change of - 
29.71. While these ships experienced a change in POTF, only three of the ships were 
determined to be statistically significant. It is important to note, however, that POTF is 
reported on a quarterly basis, and for the 48 months analyzed, there are only 16 data 
points, which is considered a small sample for analysis. 
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Table 8. ?-Test Results for POTF From CASREP 


T-THTTBMOrtvBf of TIM Free tnm CAHEP (POTTHCO^TAT) 


HuU 

MMMOf 

atMMOf 

Chanipin 

TSIatMit 

DOP 

PV^ 

L<X 


MunMr 


After 

Mmos 


CG61 

77.60 

88.63 

-11.03 

-1.20 

10.04 

0.128694 

00 

5.61 

DOG 53 

63.63 

53.00 

10.63 

0.65 

13.03 

0.737322 

00 

39.45 









. DDG 56 

Eficctivencu __ _ 

70.00 

72.50 

-2.50 

-0.16 

10.73 

0.436541 

00 

25.00 

P Value <0 0*! 

100.00 

40.13 

59.88 

5.49 

7.00 

0.999542 

oo 

80.53 

DDG 65 

87.29 

54.00 

33.29 

2.97 

9.02 

0.992218 

QO 

53.80 

DDG 69 

74.25 

64.50 

9.75 

0.67 

13.81 

0.744204 

00 

35.26 

P Value < 0.05 |Avk| 

78.79 

62.13 

16.67 

1.40 

10.61 

0.673087 

00 

39.94 

CG67 

42.29 

72.00 

-29.71 

-1.88 

10.78 

0.043357 

oo 

-1.35 

CG70 

78.25 

37.00 

41.25 

2.44 

13.99 

0.985796 

oo 

70.99 

CG72 

96.67 

72.75 

23.92 

2.34 

8.35 

0.976894 

00 

42.83 

CG73 

71.00 

48.50 

22.50 

0.99 

7.60 

0.823758 

oo 

65.04 

DDG 55 

9.25 

41.63 

-32J8 

-2.02 

12.45 

0.032608 

oo 

-3.92 

^lowance ^ 

Eliectivcness 

96.14 

73.75 

22.39 

3.22 

8.84 

0.994615 

oo 

35.17 

P Value >0.05 DDG 61 

72.00 

80.63 

-8.63 

-0.53 

6.12 

0.306526 

00 

22.73 

DDG 62 

67.71 

60.63 

7.09 

0.43 

12.12 

0.663242 

(£ 

36.33 

DDG 68 

80.80 

70.75 

10.05 

0.58 

10.79 

0.713115 

oo 

41.22 

DDG 70 

89.60 

74.63 

14.98 

1.50 

10.60 

0.919007 

00 

32.93 

DDG 73 

68.50 

82.13 

-13.63 

-0.66 

3.94 

0.273569 

00 

30.72 

DDG 76 

59.33 

78.38 

-19.04 

-1.93 

8.88 

0.042774 

00 

-0.97 

P Value < 0.05 |Avg| 

69.30 

66.06 

3.23 

0J7 

9.54 

0.564605 

00 

30.98 

All Ships IAvkI 

72.46 

64.75 

7.71 

0.72 

9.90 

0.600766 

00 

33.97 


Overall, based on the minimal changes in CASREP frequency and the lack of a 
statistically significant change of POTF, CIES-TAT appears to have had a negligible 
impact on the number of CASREPs experienced and the amount of time CASREPs are 
open. Additionally, ships that saw an increase in allowance effectiveness as a result of 
CIES-TAT did not see a reduction in CASREP frequency or marked improvement in 
POTF. The inclusive statistical relationship could be explained by an assumption that the 
processing of CASREPs onboard ships is often riddled with variable subjectivity 
introduced by the controllers of the process. 
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3. 


Impact of CILS-TAT Spending 


To measure the impact of the dollars spent on CILS—TAT, and the respective 
changes in allowance effectiveness, we turn to the amount of money obligated. Table 9 
shows a breakdown of the actual dollar amounts obligated in the year that CILS—TAT 
was implemented. These values have been adjusted to constant fiscal year (FY) 2003 
(base year) dollars using the OPN inflation category found in the 2012 Joint Inflation 
Calculator. FY2003 was chosen as the base year in conjunction with the beginning of the 
first CILS—TAT analysis periods. 


Table 9. CILS-TAT Obligation Amounts 


Hull 

Number 

Year 

Obligated 

Inflation 

Factor 

Actual 

Obligated Value 

Actual Obligated 
Value ($FY2003) 

CG61 

2005 

0.9537 

$234,668.92 

$223,801.14 

CG 67 

2006 

0.9250 

$161,646.56 

$149,525.24 

CG 70 

2007 

0.9007 

$184,329.58 

$166,024.67 

CG 72 

2005 

0.9537 

$229,774.69 

$219,133.56 

CG 73 

2005 

0.9537 

$292,576.74 

$279,027.18 

DDG 53 

2006 

0.9250 

$837.01 

$774.25 

DDG 55 

2006 

0.9250 

$751.57 

$695.21 

DDG 56 

2007 

0.9007 

$92,066.71 

$82,924.00 

DDG 59 

2005 

0.9537 

$201,782.00 

$192,437.25 

DDG 60 

2006 

0.9250 

$119,360.84 

$110,410.38 

DDG 61 

2005 

0.9537 

$182,834.45 

$174,367.18 

DDG 62 

2006 

0.9250 

$318,133.93 

$294,278.16 

DDG 65 

2006 

0.9250 

$227,216.30 

$210,178.13 

DDG 68 

2005 

0.9537 

$62,657.90 

$59,756.14 

DDG 69 

2007 

0.9007 

$86,102.12 

$77,551.72 

DDG 70 

2005 

0.9537 

$159,333.84 

$151,954.91 

DDG 73 

2005 

0.9537 

$148,749.17 

$141,860.43 

DDG 76 

2005 

0.9537 

$229,568.54 

$218,936.96 


$2,932,390.87 

$2,753,636.52 


For purposes of comparison. Figure 11 brings together the allowance 
effectiveness t-test results and the dollar amounts obligated after CILS-TAT. FY2003 
obligated values were plotted against the observed change in allowance effectiveness 
means and the respective t-test p-value. Ships that were identified as statistically 
significant are aimotated by red circles. 
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Figure 11. Explaining the Relationship Between Amounts Obligated and 
the Signifieanee of the Change in Allowanee Effeetiveness for Eaeh Ship 


The top graph in Eigure 11 shows how the three variables interaet. The bottom 
eharts are two-dimensional projeetions of the various variables from the top ehart. 
Starting with the amount obligated versus the ehange in means, while a strong 
relationship between these two does not appear to exist, using eluster analysis and the k- 
means funetion in R Statistics software, the ships were grouped into three clusters with 
similar characteristics. Eifteen parameters were used as variables in the cluster analysis, 
including obligated amount, p-values, and changes in means for the various source codes. 
These variables were used in an effort to find those parameters that differentiate the ships 
that experienced a statistically significant difference in means and those that did not. A 
grouping of three clusters was used because it naturally split the t-test p-values into three 
levels of Type I errors, with one group centered at approximately 0.05. 
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Based on this analysis, Cluster 1 (blue ellipse) eneompasses all ships with a Type 
I error approximately less than 0.05. Cluster 1 is centered on a p-value of 0.03, a change 
in means of-10.11%, and an obligated amount of $147,138. Within this cluster, the ships 
can be split again into two groups based on the amount that was obligated. In the first set 
(bottom left), DDG 53 and DDG 69 achieved a significant change in means with an 
obligated amount almost half of the other statistically significant ships—less than 
$100,000. Included in this grouping, DDG 56 did not see as large of a change in means 
for a similar amount obligated, but the change was enough to be statistically significant. 

In the second grouping in the first cluster (top left) are CG 61, DDG 59, and DDG 
65, with amounts obligated ranging from $174,000 to $210,000. They achieved a smaller 
change in the means of allowance effectiveness, with an average change of-10.63%. The 
other two clusters of ships. Cluster 2 (red ellipse) and Cluster 3 (green ellipse), 
encompass the remaining ships whose change in allowance effectiveness was not below 
the p-value threshold of less than 0.5. The important takeaway from this analysis is that 
the amount of money obligated appears to be independent of a change in allowance 
effectiveness. This is true in terms of both magnitude of the change and its statistical 
significance. For roughly the same amount obligated. Clusters 1 (blue ellipse) and 2 (red 
ellipse) achieved vastly different changes in means and statistical significance. 

Cluster analysis was also used to examine individual source codes. There are two 
for Cluster 1, source codes A and C, for which almost all ships experienced a significant 
change. Figure 12 shows the relative magnitude of the change in means and their 
statistical significance for the three clusters. Note that the values in Figure 12 have been 
normalized to allow for better comparison between variables with dissimilar scales. It is 
important to note that in terms of source codes (see Figure 4) A, C, D, and F, the 
expected change in means is negative, and for source codes G and J, the expected change 
is positive. This indicates that for the first four source codes of A, C, D, and F, the lower 
the value for change in means, the better, and for the final two source codes of G and J, 
the larger the value, the better. For the ships that are statistically significant. Cluster 1, it 
is clear that these ships experienced the largest changes in four out of six source codes; 
however, only source codes A and C experienced Type I errors less than 0.05. 
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Source Cock' Description 


Standardized Allowance Effectiveness and Variable Clusters Means 



Cluster Location & 95% Confidence Ellipse 
Cluster 


Source Code A 

Source Code C 

„,, , Souree Code D 
P Value 

Source Code F 
Source Code G 
Source Code J 


0J57258 

0.024020 

0.689032 

0.128745 

0.468553 

0.956611 

0.820998 

0.358858 

0.842876 

0.543827 

0.022968 

0.565930 

0.063805 

0.936188 

0.561863 

0.026774 

0.769961 

0.036699 


0.333200 

0.014703 

0.278194 

0.177577 

0.455850 

0.076607 

0.268588 

0.334407 

0.209272 

0.362566 

0.009545 

0.40240<> 

0.081574 

0.0637()4 

0.397449 

0.066252 

0.216851 

0.087970 


P Value: Allowanee Effectiveness 
Change in Means: Allowance 


Source Code A 
Source Code C 
Change in Souree Code D 
Means Source Code F 
Souree Code G 
Source Code J 


U.00936I 

10.26 

18.09 

5.99 

-15.83 

0.05 

-5.80 

-39.96 


0.014639 

3.05 

31.28 

3.63 

12.53 

3.53 
4.32 
18.19 


0.139951 

5.99 

34.64 

3.09 

4.55 

4.92 

5.38 

13.83 


0.378823 

1.58 

-16.49 

-5.52 

-13.09 

-0.39 

1.07 

-28.17 


0.111190 

3.39 

4.13 

5.47 

9.92 

I. 41 
3.34 

II. 78 


0.251522 

2.79 

29.62 

4.10 

12.39 

2.61 

4.31 

12.01 


Allowance List Material issues | 
Ironi storeroom stock 


Non-Allowance List Material 
(SI.M I issues from storeroom sUKk 


Not Carried (NO repair parts 
which are listed on an APL. but 
docs not compute for an allowance 


Allowance List Material Not In 
Stock (NISI when requested 

Non-Allowance List Material 
(SI.M) Not In Stock (NIS) when 
requested 

Not Carried (NO repair pans 
which arc not listed on an APL in 
the ship's COS file 


Figure 12. Source Codes That Drive Allowance Effectiveness to be 
Statistically Significant Using Cluster Analysis 


Looking at Table 10, which shows the results of t-test analysis for the various 
source codes in conjunction with Figure 13 in terms of its change, source code C had the 
lowest p-value both individually and when averaged across the six statistically significant 
ships. However, in terms of numerical magnitude and the parameters’ relative ability to 
change the allowance effectiveness equation, the primary parameter of concern is source 
code A. This can be seen in the ellipse for change in source code A, which is centered 
on -57.73, while the ellipse for source code C is centered on -12.18. In summary, it was 
found that source code A had a greater ability to impact the change in allowance 
effectiveness than any other source code 

In keeping with this theme, overall changes in source code G are not statistically 
significant within a 95% confidence interval. This can be explained by looking at the t- 
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test results for souree eode G. Looking at the p-values for the greater than and less than 
hypothesis, there does not appear to be a clear direction among all of the six ships with 
statistically significant change nor among all ships for the change in source code G; out 
of all of the ships, 33% were statistically significant with a greater than hypothesis and 
17% with a less than hypothesis. In terms of source code J, the ships in Cluster 1 fell 
between the other clusters. Similarly, this is explained by the ambiguous direction of the 
change in means among all ships; for the greater than hypothesis, 17% of the ships 
experienced a Type I error less than 0.05, and likewise, under the less than hypothesis, 
17% of the ships experienced a Type I error less than 0.05. Essentially, the results of the 
G and J source codes are not consistent and not statistically significant. The data is split 
unevenly in directionality. Early on, CIES—TAT affected A and C but did not affect the 
G and J source codes. According to the framework of source codes, you would expect 
some correlation between A/C and G/J. As stated previously, the data analysis does not 
reveal any correlation between the four source codes of A, C, G, and J. 

These changes are consistent with the expected results of CIES—TAT 
implementation since the technique is designed to increase the frequency of source codes 
A and C. If demands received in the after period are identical to demands received in the 
before period, then one would expect the occurrence of source codes A and C to increase 
while G and J decreased in a perfectly inverse relationship. In reality, however, there are 
other factors that influence the frequency of G and J that cannot be completely mitigated 
by CIES—TAT (e.g., poor configuration management practices). This proves that, 
independent of the amount of money obligated, in those ships that achieved a statistically 
significant change in allowance effectiveness, CIES—TAT was able to accurately target 
source codes A and C. This implies that the true measure of CIES—TAT’s impact on 
allowance effectiveness can be quantified simply by focusing analysis on changes in 
source codes A and C. 
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Table 10. Analysis of Source Codes for Allowance Effectiveness Calculation 


[P VUuc]: AUowmce EfTectiveMts Sovce Codes Analyris |CILS>TAT Period) 

AUowwc SiMirtc Csif A Shib Csdt C Sooree Cede D Soair* Cede F Seen* Cede G Seam Cede J 


BmU 

Nuibcr 

Greater 

Lcaa 

Greater 

Leaa 

Greater 

Uaa 

Greater 

Leaa 

Greater 

Lan 

Greater 

Uaa 

Greater 

Lem 

CG 61 

0.9999 

0.0001 

0.9998 

0.0002 

0.9984 

0.0016 

0.9306 

0.0694 

0.9996 

0.0004 

0.5157 

0.4843 

02295 

0.7705 

DD0 53 

0.9988 

0.0012 

0.9994 

0.0006 

0.9997 

0.0003 

0.9951 

0.0049 

0.6573 

0.3427 

0.0036 

0.9964 

0.8045 

0.1955 

0.9501 

0.0499 

0.9883 

0.0117 

1.0000 

0.0000 

l.OOOO 

0.0000 

0.6505 

0.3495 

0.0546 

0.9454 

0.9752 

0.0248 

0.9951 

0.0049 

0.9999 

0.0001 

0.9697 

0.0303 

0.7836 

0.2164 

0.4236 

0.5764 

0.9547 

0.0453 

0.6293 

0.3707 

<0.05 “““ 

0.9933 

0.0067 

0.9927 

0.0073 

1.0000 

0.0000 

0.0031 

0.9969 

0.0773 

0.9227 

0.0015 

0.9985 

0.2009 

0.7991 

DDG69 

1.0000 

0.0000 

0.8328 

0.1672 

1.0000 

0.0000 

0.9991 

0.0009 

0.0348 

0.9652 

0.0002 

0.9998 

0.0012 

0.9988 

P Value < 0.05 (Avcraite) 

0.9895 

0.0105 

0.9688 

0.0312 

0.9946 

0.0054 

0.7852 

0.2148 

0.4738 

0J262 

0JS50 

0.7450 

0.4734 

0i>266 

CG67 

0.8848 

0.1152 

0.4458 

0i542 

1.0000 

0.0000 

0.0059 

0.9941 

0.9928 

0.0072 

0.0133 

0.9867 

0.1142 

0.8858 

CG 70 

0.0870 

0.9130 

0.6767 

OJ233 

0.6379 

0.3621 

0.9818 

0.0182 

0.9958 

0.0042 

0.9230 

0.0770 

0.9925 

0.0075 

CG72 

0.7416 

0-2584 

0.9361 

0.0639 

0.0000 

1.0000 

0.8464 

0.1536 

0.0841 

0.9159 

0.9038 

00962 

0.3629 

0.6371 

CG73 

0.9391 

0.0609 

0.9982 

0.0018 

0.9886 

0.0114 

0.7845 

0.2155 

0.7052 

0.2948 

0.7088 

0.2912 

0.7205 

0.2795 

Allowance DDG 55 

0.3263 

0.6737 

0.9453 

0.0547 

0.0146 

0.9854 

0.9935 

0.0065 

0.9979 

0.0021 

0.7838 

0.2162 

0.9932 

0.0068 

rffintilSil DDG 60 

0.8793 

0.1207 

0.8238 

0.1762 

1.0000 

0.0000 

0.9614 

0.0386 

0.9986 

0.0014 

0.0297 

0.9703 

0.8825 

0.1175 

P Value DDG 61 

0.7788 

02212 

0.0087 

0.9913 

0.4629 

0.5371 

0.9959 

0.0041 

0.0945 

0.9055 

0.5556 

0.4444 

0.0114 

0.9886 

> 0.05 DDG 62 

0.8042 

0.1958 

0.7777 

02223 

0.3254 

0.6746 

0J124 

0.6876 

0.9674 

0.0326 

0.0007 

0.9993 

0.8388 

0.1612 

DDG6K 

0.1715 

0.8285 

0.4130 

OiSTO 

0.6932 

0.3068 

0.2228 

0.7772 

0.0001 

0.9999 

0.1321 

0.8679 

0.7680 

0.2320 

DDG 70 

0.5246 

0.4754 

0.9993 

0.0007 

0.4345 

0.5655 

0.9936 

0.0064 

0.9851 

0.0149 

1.0000 

0.0000 

0.7420 

02580 

DDG 73 

0.6806 

0J194 

0.2597 

0.7403 

0.6250 

0.3750 

0.0443 

0.9557 

0.9322 

0.0678 

0.5000 

0.5000 

0.0714 

0.9286 

DDG 76 

0.8746 

0.1254 

0.4467 

Oi533 

0.1361 

0.8639 

0.2098 

0.7902 

0.0003 

0.9997 

0.9999 

0.0001 

0.0012 

0.9988 

P Value > 0.05 lAvst^ 

0.6410 

0J59O 

0.6442 

0JS58 

0.5265 

0.4735 

0.6127 

0.3873 

04>462 

0J538 

0.5459 

0A541 

0.5415 

0.4585 

% of Ships Statistically 
Signiftcaiu Ships with 

OK 

lOOK 

OK 

B3K 

OK 

lOOK 

17K 

50K 

17K 

17K 

50K 

17K 
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17K 

PVal<0.05 
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Figure 13. Statistical Significance of Change in Allowance Effectiveness 

(Before to After CILS—TAT) 


The graphs in Figure 13 depict the relationship between the observed changes in 
data set means and their statistical significance relative to the p-value and the 95% upper 
confidence level (UCE). The UCL would be defined in this case as a type of interval 
estimate of a population parameter and is used to indicate the reliability of an estimate. 
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These two graphs show that of the data sets, which did not show statistical 
significance, there are some ships (Cluster 2) relatively close to achieving statistical 
significance. Relating Figure 13 back to Table 10, if the confidence interval was 
increased to 99%, the number of statistically significant ships decreases to five (one CG 
and four DDGs). However, in order to increase the number of statistically significant 
ships to 90% of all ships (~16 ships), the confidence interval must be changed to 30%, 
which is well outside of acceptable limits. Again, statistically significant means that at a 
95% confidence interval (Type I error < 0.05) there is enough evidence, based on the 
change in means and the variance in the two data sets, to say that there was a change. 
Note that there are three ships—CG 70, DDG 55, and DDG 68—that experienced a 
positive change in means, which is opposite to the expected output. Herein, this shows 
that while there are some ships on the cusp of achieving statistical significance, only 
marginal losses or gains are made by increasing or decreasing the confidence interval— 
the ships that achieved statistical significance were well inside the 95% confidence 
interval. 

In conclusion, during the periods when CILS—TAT was implemented, there were 
six ships that experienced a significant enough change in allowance effectiveness to 
result in a Type I error less than 0.05. While the amounts obligated for each ship varied 
significantly, there appears to be no relationship between the amount obligated and the 
resulting change in allowance effectiveness. This result is consistent for both ships that 
experienced a statistically significant change and those that did not. In terms of the 
parameters used to calculate allowance effectiveness, CILS—TAT predominately 
impacted source codes A and C with some change to codes G and J; however, not in a 
statistically significant manner. 

4. Brownout Period Analysis 

Data analyzed during the brownout period covers December 2009 to January 
2011. In order to maintain uniformity of CILS—TAT effectiveness for those ships whose 
24-month window continued past December 2009, their brownout analysis window is 
adjusted. For example, USS John S McCain's brownout period started as of June 2009, 
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since the 24 months following CILS—TAT implementation did not complete until June 
2009. This ensures that all ships’ data are given the full 24-month window following 
CILS—TAT implementation. 

During the brownout period, there was a considerable decrease in allowance 
effectiveness across the board. Looking at the two data sets within the before data, the 
reported allowance effeetiveness ranges from a minimum of 25.30% to a maximum of 
98.40% with an overall average of 71.12% and a standard deviation of 13.28%. In terms 
of ships grouped by class, there is not an appreciable difference in the before data sets. 
Among the after data sets, allowance effectiveness ranges from a minimum of 0.00% to a 
maximum of 100.00% with an overall average of 66.35% and a standard deviation of 
13.96. Table 11 summarizes the ehanges in the means data sets from before to after. To 
be consistent for comparison purposes with the change in means caleulated in the Welch 
Two-Sided t-test, the change in means calculation is based on the before data set mean 
minus the after data set mean. 


Table 11. Summary of Allowance Effectiveness (Before to After Brownout Period) 
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In terms of statistical significance for the brownout period, the expected result 
was that the mean of the before data set would be greater than the mean of the after data 
set. Based on this expected movement, the greater hypothesis was used as the base for 
analyzing the brownout period. Again using the Welch Two-Sided Mest, 10 ships (four 
CGs and six DDGs) saw a statistically significant change in the means of the before and 
after data sets. On average, all of the ships experienced a change of 4.77. When grouped 
together, CGs and DDGs experienced an average change of 6.72 and 4.03, respectively, 
in the data sets. 

In stark contrast to the change experienced during the GILS—TAT period, during 
the brownout period, at a confidence interval of 99% (Type 1 error < 0.01), there is 
enough evidence to prove that six ships experienced the expected change in allowance 
effectiveness. There are, however, four ships that experienced a small, negative change 
in mean, resulting in very low statistical significance. For the brownout period, in order 
to capture 90% of the ships (~16 ships) being statistically significant, the confidence 
interval would need to shift to 60%. Table 12 summarizes the t-test results. 


Table 12. t-Test Summary of Allowance Effectiveness 
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Cluster analysis was again used to discover the reason that these 10 ships 
experienced a statistically significant change. Figure 14 shows that during the brownout 
period, ships in Cluster 1, similar to the CILS—TAT period results, had the largest 
changes in source codes A, C, G, and J. However, during the brownout period, source 
codes G and J—vice A and C—had a Type I error less than 0.05. Looking at the 
magnitude of the change in means among the source codes, it is clear that source code J 
dominated the movement of allowance effectiveness. For comparison, source code J’s 
ellipse was centered on a change in means of -1.31 during the CILS—TAT period and on - 
39.96 during the brownout period; source code A’s ellipse was centered on -57.73 during 
the CILS—TAT period and on 18.90 in the brownout period. During the brownout period, 
both the magnitude and statistical significance flipped. Looking at the parallel coordinate 
plot at the top of the figure, note that during the brownout period, the higher the value, 
the better for the change in means of source codes A, C, D, and F and the smaller the 
number, the better for the change in means of source codes G and J. Based on this 
analysis, it is clear that the change in source code J is the reason that the 10 ships 
experienced a statistically significant change in allowance effectiveness. 
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Source Code Description 


Standardized Allowance Effectiveness and Variable Clusters Means 
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Figure 14. Impact of Source Codes on Allowance Effectiveness Using Cluster Analysis 

In conclusion, it is clear that the brownout period caused an impact on a larger 
number of ships when compared to the CILS—TAT period. Flowever, looking at the 
bigger picture and Figure 15 which shows allowance effectiveness and the standard 
deviation for the three different periods, it is clear that the brownout period resulted in 
many of the ships returning to their pre-CILS—TAT means and standard deviations. 
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Variance in Allowance Effectiveness Before to After Brownout 


Indicates statistically significant ships at 95K confidence (a < 0.05) 
Before CILS-TAT - After CILS-TAT Brownout 



Figure 15. During the Brownout Period, Many Ships Return 
to Before CILS-TAT Levels 


In summary, five ships were statistieally significant during both the CILS-TAT 
period and the brownout period. Of those five ships, three ended up with allowance 
effectiveness means that were higher than their pre-CILS—TAT levels, indicating a 
lasting effect of the implementation. Six ships were not significant during the CILS- 
TAT analysis but returned to within 10% of their pre-CILS—TAT mean during the 
brownout phase. This change is not directly related to CILS-TAT, although it is a 
variable to consider. The remaining seven ships did not see a significant change in mean 
in either of the analysis windows. 
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V. CONCLUSIONS AND RECOMMENDATIONS 


A. CONCLUSIONS 

In summary, the goal of this research was to conduct a BCA of CILS—TAT to 
determine its historical effectiveness against allowancing metrics. The analysis examined 
the before and after periods to determine if the benefits were substantial enough to restart 
the program following the implementation of the Navy’s ERP. The research identified 
three specific questions used in guiding the process. In the following paragraphs, we 
summarize our findings. 

First, we asked the question of how did CILS—TAT impact allowance 
effectiveness for our sample and at what cost? The findings suggested that CILS—TAT 
had a direct and positive impact on allowance effectiveness for six out of 18 ships. Of 
the 12 remaining ships, nine saw a positive change in allowance effectiveness, although 
this change is not statistically significant and thus cannot be attributed to CILS—TAT. 
Based on these findings, CILS—TAT does achieve the ends for which it is designed. 
Regardless of whether the U.S. Navy re-implements this program or some other targeted 
allowancing technique, we have found that there is a need throughout the fleet for this 
model. 

Second, we explored the question of whether there are correlations between 
allowance effectiveness rates, POTF from CASREPs, and CILS—TAT cost. Our research 
concluded that there was, in fact, no direct correlation between the effectiveness of 
CILS—TAT and these three variables. Of the ships that were statistically significant, their 
performance as related to allowance effectiveness and POTF could not be linked directly 
to the amount of funds obligated. 

Last, we sought to answer the question of the effect of not having CILS—TAT 
during 2009-2011, when traditional allowancing procedures were not available. The 
research demonstrated that in the absence of targeted allowancing techniques, ships that 

received CILS—TAT were able to maintain allowance effectiveness rates within 10% of 
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their pre-CILS—TAT averages for up to 24 months following implementation. These 
results are most encouraging since they display the longer term impact of CILS—TAT as 
well as provide an expected shelf life for each iteration of the process. 

Upon concluding our research, we were able to determine that CILS—TAT did 
directly contribute to the improved allowance effectiveness of ships within the BMD 
fleet. Ships in the sample that saw increases in allowance effectiveness were able to 
maintain those higher levels of performance for up to two years in both the presence and 
absence of traditional allowance maintenance processes. While CILS—TAT did not 
improve allowance effectiveness to the same extent on every ship, the fact that it did 
improve one third of our sample cannot be ignored, and we recommend that the Navy 
continue to utilize this program to improve allowance effectiveness in the future. 

1. Recommendations for Change 

How can the effectiveness of a CILS—TAT product be measured? This depends 
on the priority of the decision-maker as to whether CILS—TAT should be used to 
primarily improve allowance effectiveness with a second order effect of reducing the 
number of CASREPs or whether the program should first reduce CASREP frequency and 
then focus on allowance effectiveness if resources exist. We believe that the system 
could be tailored in either direction. 

CIES—TAT seeks to improve allowance effectiveness first with no consideration 
given to the priority of critical systems. Allowances added as a result of CIES—TAT are 
funded on an all-or-nothing basis, and when funding is constrained, use of a knapsack 
model would aid in optimizing dollars spent. For example, funding could be applied to 
all 9X Cognizance Code (COG) items with MCCs 2, 3, or 4 first since this family of 
COGs accounts for 34.76% of all CASREPs recorded and only 3.02% of total CASREP 
cost. Additionally, MCCs 2, 3, and 4 accounted for 75.17% of all CASREPs during the 
analysis window. If more priority were given to these MCCs, the same effect on overall 
system readiness could be realized at half of the cost. Over the time period analyzed, 

CILS—TAT requisitions were funded at the rates shown in Table 13: 
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Table 13. Breakdown of CILS—TAT Requisition Cost (2003-2009) 


CELS-TAT Requisition Cost 
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As shown in Table 13, nearly 40% of all funding applied as a result of CILS— 
TAT is used for NIINs with MCC 1 (failure results in minor mission impact). By using a 
nearly identical amount of funding across MCCs 2, 3, and 4, a much larger impact could 
be realized on the ship’s ability to perform its mission. 


40.00% 
35.00% - 
« 30.00% - 

O 

8 25.00% - 
■5 

S, 20.00% 

i 

5 15.00% - 
a 

t 10 . 00 % 
5.00% 






0.00% -i- 

.9 COG 

1 

4.90% 

2 

10.69% 

3 

15.49% 

4 

8.57% 

N/A 

4.55% 

-7 COG 

6.37% 

7.66% 

16.77% 

11.94% 

7.37% 

■ 3 COG 

0.47% 

1.28% 

1.14% 

0.69% 

0.41% 

_1C0G 

0.49% 

0.27% 0.52% 0.14% 

Mission Cnticallity Codes (MCC) 

0.27% 


Figure 16. Breakdown of CASREP Frequency 
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Figures 16 and 17 clearly demonstrate the effect of making a change in funding 
priorities by using MCCs as a factor. Results are based on the analysis of 13,956 
CASREPs submitted by sample ships during the analysis window. 

2. Potential Obstacles 

The primary obstacle to changing the implementation of CILS—TAT would be in 
making the shift away from allowance effectiveness as a primary measure, to an approach 
focused more on critical systems. In a perfect world, all systems would operate as 
required at all times, but in reality, certain systems will experience difficulty and operate 
in a degraded capacity. For this reason, it is important that we focus more closely on 
critical systems both during the initial allowance provisioning process and during times 
when allowances are being maintained. By shifting to an MCC-based allocation model, 
we would be moving away from some of the more traditional supply effectiveness 
metrics. Since this change would be both cultural and procedural in nature, it makes 
implementation the most difficult. 
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This change would result in a decrease in allowanee effeetiveness but an inerease 
the ability of a ship to earry out mission tasking. In an era where the DoD is expeeted to 
operate at prior years’ levels with fewer resourees, perhaps this cultural change is 
neeessary. 

B. RECOMMENDATIONS FOR FUTURE RESEARCH 

During the eourse of our researeh, we identified some areas where our researeh 
eould be improved or additional questions eould be answered through the use of a larger 
sample size as well as a more diverse set of platforms seleeted. An additional 
improvement to this researeh would be to seleet a eontrol group, or in this ease, a control 
ship. This ship would be evaluated over the same time period; however, it would not 
reeeive a CILS—TAT. Its performanee eould then be eompared against several other 
ships of the same elass or mission area to determine the long-term benefits of the 
program at the unit level. 

There are other variables that eould also eontribute to the supply effeetiveness of 
a ship that were not evaluated here. Partieularly, if supply gross and net effeetiveness are 
to be evaluated, then operational funding has to be eonsidered as well. Improved 
operational funding levels will inerease the opportunity for a ship to replenish its stoek 
and thereby reduee the number of Not in Stoek (NIS) demands. Our researeh evaluated 
the NC rate for eaeh ship and was therefore independent of periodie OPTAR funding 
grants. 

A study should be conducted eomparing the allowanee maintenanee done through 
traditional proeesses as well as through CILS—TAT. We were unable to gain aeeess to 
this data, but we hypothesize that those ships in our study that did not see a signifieant 
statistieal ehange attributed direetly to CILS—TAT may have been a result of inereased 
traditional allowance produets (e.g., more allowanee adds through ASIs). The results of 
this new study would provide a more aecurate understanding as to the preeise impact of 
CILS-TAT. 
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Lastly, in order to grasp CILS—TAT’s impact on CASREP frequency, additional 
analysis could be conducted on NIINs added as a result of CILS—TAT. Since the 
allowances added to a ship following a CILS—TAT are derived from previously NC 
demands, a percentage of those demands were recorded as CASREPs. Analysis could be 
conducted to determine the demand frequency of previous CASREP NIINs that are added 
to the ship’s stock and subsequently avoid a new CASREP. 


72 



LIST OF REFERENCES 


Anderson, D. R., Sweeney, D. J., & Williams, T. A. (2000). Essentials of statistics for 
business and economics. Cincinnati, OH: South-Western. 

Axinto, M. I., & Giles, M. S. (2005). Review and analysis of selected items management 
(SIM) inventory program aboard US surface ships (Master’s thesis. Naval 
Postgraduate School). Retrieved from http://handle.dtic.mi1/100.2/ADA442828 

Alvarez, R. (2010). Reducing the logistics footprint in naval ships through the 
optimization of allowance equipage lists (AELs) (Master’s thesis. Naval 
Postgraduate School). Retrieved from http://www.dtic.mil/cgi- 
bin/GetTRDoc?AD=ADA524564 


Boardman, A. E. (1996). Cost benefit analysis: Concepts and practice. Upper Saddle 
River, NJ: Prentice Hall. 

Brealey, R. A., Myers, S. C., & Allen, F. (2011). Principles of corporate finance. New 
York, NY: McGraw-Hill/Irwin. 

Bruno, J. (1999, February). Continuous ILS targeted allowance technique [Presentation 
slides]. Naval Inventory Control Point, Mechanicsburg, VA. Code 055X. 

Chief of Naval Operations (CNO). (2003). Operational availability of equipment and 
weapons systems (OPNAVINST 3000.12A). Retrieved from Department of the 
Navy Issuances System website: http://doni.daps.dla.mil/Directives/03000 
%20Naval%200perations%20and%20Readiness/0300%20General%200peration 

s%20and%20Readiness%20Support/3000.12A.pdf 

Chief of Naval Operations (CNO). (2012). Approval and funding for shore-based retail 
allowances. Retrieved from Department of the Navy Issuances System website: 
http://doni.daps.dla.mil/Directives/04000%20Fogistical%20Support%20and%20S 

ervices/04-400%20Supplv%20and%20Material%20Services/4441.13B.pdf 

Christofides, N., Mingozzi, A., Sandi, C., & Toth, P. (1979). Combinatorial optimization. 
New York, NY: Wiley. 

Configuration Data Managers Database-Open Architecture (CDMD-OA). (2012). 
Homepage. Retrieved from http://www.cdmd.navv.mil/ 

Defense Acquisition University (DAU). (2012). Total ownership cost. Retrieved from 
https://acc.dau.mil/CommunitvBrowser.aspx?id=204301 

Defense Acquisition University (DAU). (2013). Business case analysis. Retrieved from 
https://acc.dau.mil/bca 


73 













Defense Systems Management College. (1997, Deeember). Acquisition logistics guide 
(3rd ed.). Fort Belvoir, VA: Government Printing Office. 

Department of the Navy (DoN). (2009). COSAL use and maintenance manual revisions 
(NAVICPINST 4441.170B). Retrieved from Navy Knowledge Online website: 
https://wwwa.nko.navv.miFportal/navsea/navseaips/home/40supplvsupport 

Department of the Navy (DoN). (2012). Department of the Navy fiscal year (FY) 2013 
budget estimates. Retrieved from 

http://www.finance.hq.navv.mil/fmb/13pres/OMN Voll book.pdf 

General Accounting Office (GAO). (2003, August). Defense inventory: Opportunities 
exist to improve spare parts support aboard deployed navy ships (GAO-03-887). 
Retrieved from http://www.gao.gov 

Government Accountability Office (GAO). (2010, July). Defense management: DOD 
needs better information and guidance to more effectively manage and reduce 
operating and support costs of major weapons systems (GAO-10-717). Retrieved 
from http://www.gao.gov 

Government Accountability Office (GAO). (2012, September). Military readiness: Navy 
needs to assess risks to its strategy to improve ship readiness (GAO-12-887). 
Retrieved from http://www.gao.gov 

Harrahy, D. J., Powell, R. I., & Lutz, R. (1968, August). Optimum cost shipboard 

allowance lists for maintenance parts. Naval Engineers Journal, 80{4), 577-582. 

Keller, G. (2009). Statistics for management and economics. Mason, OH: South-Western 
Cengage Learning. 

Lind, D. A., & Mason, R. D. (1993). Statistical techniques in business and economics. 
Burr Ridge, IL: Irwin. 

Missile Defense Agency, (n.d.). Our mission. Retrieved from 
http://www.mda.mil/about/mission.html 

Naval Sea Logistics Center (NSLC). (2012). Mission and vision. Retrieved from 
http://www.nslc.navsea.navv.mil/htm/nslc/mission vision.htm 

Naval Supply Systems Command (NAVSUP). (2010). Navy ERP “live” at Naval Supply 
Systems Command. Retrieved from http://www.navv.mil/submit/displav. 
asp?storv id=51986 

Martello, S., & Toth, P. (1990). Knapsack problems: Algorithms and computer 
implementations. New York, NY: J. Wiley & Sons. 


74 











Naval Supply Systems Command (NAVSUP). (1997). Naval supply procedures 

(NAVSUP Publication 485). Retrieved from Navy Basic Military Requirements 
website: http://www.navvbmr.com/NAVSUP%20P-485.html 

Nemhauser, G. L., & Wolsey, L. A. (1988). Integer and combinatorial optimization. New 
York, NY: Wiley. 

Neter, J., Wasserman, W., & Kutner, M. H. (1990). Applied linear statistical models: 

Regression, analysis of variance, and experimental designs (3rd ed.). Homewood, 
IL: Irwin. 

Noether, G. E. (1971). Introduction to statistics. Boston, MA: Houghton Mifflin. 

Office of Management and Budget (0MB). (1992). Guidelines and discount rates for 
benefit-cost analysis of federal programs. Washington, DC: Author. 

Office of Management and Budget (0MB). (2003, May). Department of the Navy FY 
2003 budget estimates budget data book. Retrieved from 
http://www.finance.hq.navv.mil/fmb/03pres/FY03 DataBook.pdf 

President of the United States. (2010). National security strategy. Retrieved from 

http://www.whitehouse.gov/sites/default/files/rss viewer/national security strate 

gy.pdf 

Ragsdale, C. T. (2008). Spreadsheet modeling & decision analysis: A practical 

introduction to management science. Mason, OH: Thomson/South-Westem. 

Sykes, A. (1993). An introduction to regression analysis. Chicago, IL: University of 
Chicago Law School. 

U.S. Navy Enterprise Resource Planning (ERP). (2013). About Navy ERP. Retrieved 
from http://www.erp.navv.mil/ 

Ye, N. (2003). The handbook of data mining. Mahwah, NJ: Lawrence Erlbaum 
Associates. 


75 







THIS PAGE INTENTIONALLY LEET BLANK 


76 



INITIAL DISTRIBUTION LIST 


1. Defense Teehnieal Information Center 
Ft. Belvoir, Virginia 

2. Dudley Knox Library 
Naval Postgraduate Sehool 
Monterey, California 


77 



